Understanding the statistics

From VPOIDS Documentation

AFIDS Statistics

Statistics are reported from AFIDS in a number of ways (metrics and dimensions) depending on the intended use of the data. Statistics may not match between reports depending on how they are intended to be used. The discrepancies are explained below.

Some statistics are generated on the fly from current operational data. Other statistics come from a special repository of "snapshot" data known as the permanent statistics. Permanent statistics are captured in order to maintain the integrity of historical data. This is like a financial audit, which "closes the books" on a period. Using current operational data can result in variances over time which are not desirable for historical data. Some of these variances are legitimate, some are the result of retroactive changes to historical data made in error. Legitimate variances result from annual updates to the aircraft cost data, pilots moving from wing to wing (which affect only wing statistics), for example.

Whenever possible, permanent statistics should be used for public, external reporting, since they will not change over time. However, since only a certain level of granularity is captured in the snapshot, it is sometimes necessary to use current operational data to generate historical statistics. When that's the case, it's sometimes necessary to footnote the numbers in case they don't match the permanent statistics.

Important notes:

  • The metric for publicly reporting the organization's activity is based on mission legs, which is a trip flown by a pilot from point A to point B.
  • Public flown counts are always based on approved mission reports. Mission legs must have an approved mission report in order to be counted.

Metrics and dimensions that affect statistics

In order to better understand the activity and impact of the organization, the statistics have several metrics and dimensions.

Statistical mission types

Most organizations have some mission types which the organization does not wish to count in its public mission counts. These typically include service missions and commercial companions. These mission types are typically excluded in almost all reports, but they are relevant for (and therefore are included in) financial reports and some operational reports, since they represent an in-kind donation, and since they represent coordination effort.

Organization legs and linking legs

Organizations only count mission legs flown by their pilots. This is the organization's leg, and other legs in the mission may be linking legs flown by other pilots. For internal reporting, the organization may wish to know how many linking legs they handled, though indirectly, since a linking leg does represent some coordination effort.

Mission leg types

There are three types of mission legs, pilot flown, commercial, and ground. These numbers may be reported publicly, or used internally.

Reports and their Purposes

The three main purposes for reports are:

  • External reporting of organization activity and impact
  • Financial reporting for taxes, etc.
  • Internal reporting for process analysis, operational metrics

External Reporting

Permanent Statistics

The permanent statistics report should be used for all external reporting donors, foundations, and for any reporting with financial implications. Permanent statistics include all mission types and all flights, both pilot flown, commercial, and ground.

Permanent statistics are generated once a month is reconciled and “closed out.”

Statistical Summary

The Statistical Summary can be used for reporting in advance of a month being “closed” and permanent statistics being generated. Using this report for months in the past is not advised, as discrepancies may arise over time (see “Troubleshooting Report Discrepancies” below).

To match the numbers on this report with the permanent statistics, make sure that the “Include” option is set to all, which will show all mission types. If the numbers do not match, then there is a problem with some of the missions which need to be corrected (see “Troubleshooting Report Discrepancies” below).

Internal reporting

Mission Counts YTD. This report should only be used for approximations of mission numbers in the near past, present and near future. It does not use mission reports for verification so that near past, present and near future missions can be considered. As a result, this report cannot be reliably reconciled with any other report.

Mission Counts by Member. This report is based on approved mission reports, but not the permanent statistics. If there is a discrepancy between this report and the permanent statistics, then there is a problem with some of the missions which need to be corrected (see “Troubleshooting Report Discrepancies” below).

Cancelled mission reports. Cancelled missions do not have a formal verification via a mission report. All reports that present cancelled missions are subject to fluctuation, especially present and near past missions. Where reports compare cancelled and flown missions, mission reports may not be considered in order to count present and near past missions, and therefore should not be compared to permanent statistics reports or other reports that count only missions with approved mission reports.

AFA Legs. AFA legs do not have mission reports, and therefore are not formally verified. It is assumed that they are flown if not cancelled.

Mission Counts YTD

This report is intended as a gauge of current activity, not as a record of ongoing statistics. These numbers should not be reported publicly.

Scheduled missions is a count of all mission legs entered into the system. If the date of the mission changes, this will affect the month-by-month count.

Cancelled is a count of mission legs with a cancel status. If the mission leg was cancelled prior to a pilot having been assigned, there is no wing designation.

Flown is simply the difference between the scheduled and cancelled mission legs. This number is subject to significant fluctuations as mission legs are cancelled.

Report approved is the count of mission legs that have an approved mission report. This number is also subject to significant fluctuations for the current month as mission reports are corrected and approved.

Reports based on Approved Mission Reports

Most reports in the system count flown mission legs based on approved mission reports.

These reports also base the monthly counts on the date of the mission report. It is assumed that this is the most accurate date. Sometimes a mission leg will move from one month to the next. For example, if a mission scheduled for the last day of the month is delayed due to weather and flown the next day, the mission report will reflect the actual date flown, i.e. the first of the next month. That mission leg will be counted in the next month whereas the scheduled date of the mission leg will not be changed.

Approved mission report number will fluctuate in the current month and for the previous month until the month is reconciled, meaning that all mission legs have been accounted for properly and all mission reports have been filed, corrected and approved.

The wing designation of a mission leg is based on the wing of the pilot. If the pilot moves wings, the mission legs he or she flew in the past will shift from the old wing to the new wing. For pilots that move seasonally from one wing to another, the count will shift back and forth as their wing designation changes. Mission cost, hours and distance figures for the purpose of wing tracking are affected similarly.

Some reports designed for publication outside the organization, like the AFA Monthly Report, are based on approved mission reports. These reports should not be published until the month’s mission reports are reconciled, and these reports should not be used for historical trends. If run after a month’s mission reports have been reconciled, but not too long after, the numbers should closely resemble the numbers in the permanent statistics reports.

Permanent Statistics

The objective of the permanent statistics is to establish a set of mission leg counts that can be reported and compared over time, and so therefore will not change once established.

Mission legs statistics are captured as a snapshot for each month once the month has been reconciled. This process is initiated by the staff after the mission reports for the month have been successfully reconciled. These statistics should be processed as soon as possible to minimize the effects of any changes that may take place. The statistics include the leg count, the hours flown, the cost (both pilot and commercial tickets) and the distance. Individual data is captured for each wing and mission type.

In the reports that use the permanent statistics data, no data for a month will appear until the month has been processed. There are no partial reports, and therefore, no moving numbers. When these numbers are reported, it must be made clear that a 0 number for the previous month’s mission legs is potentially due to the fact that the statistics have not yet been processed.

Permanent statistics are designed to be the numbers reported to the public. Because they do not change, these are useful for tracking historical trends. The permanent statistics are the basis for the numbers are reported on our public web site. Year-to-date missions on the web site only reflect months that have been reconciled and processed into permanent statistics.

Membership Numbers of active members collected on the last day of each month for each wing, for each flight status (command pilot, non-pilot, or verify orientation).

Missions Numbers of mission legs flown collected on the last day of each month for each wing, for each mission type, and separately for each cancel reason.

The permanent statistics "roll up" only specific metrics:

  • leg_count - all mission legs
  • total_hours - total hobbs time for pilot flown mission legs
  • aircraft_cost - cost for pilot flown mission legs, calculated as hobbs_time * aircraft cost
  • commercial_cost - cost of commercial tickets purchased by pilots (for pilot flown mission legs)
  • airline_donation - cost of commercial tickets provided by airline partners (commercial mission legs)
  • air_leg_count - count of pilot flown legs
  • commercial_leg_count - count of commercial legs
  • ground_leg_count - count of ground legs
  • ground_miles - miles logged on ground legs

The following dimensions are available:

  • Year and month
  • Mission type
  • Wing

ground_cost` int(11) NOT NULL DEFAULT '0',

Member data

AFIDS does not explicitly track when a member is “de-activated” and therefore it is not possible to get month by month counts of how many members belong to the organization. Note that a member is considered active even after his or her renewal date has passed, until he or she has received all four reminder letters. After the final letter is sent (approximately three months after the renewal date), the system sets the member to inactive status.

AFIDS now captures a snapshot of the counts of members in each flight status category on the first day of each month. The number reported for the month is therefore the count at the beginning of that month.

Passenger Age

For the purpose of statistics, a passenger’s age is calculated based on their birth date and compared to the date of the mission. A passenger is considered to be a child if he or she is 18 years or under at the time of the mission. In other words, the mission date is prior to the passenger’s 19th birthday.

Troubleshooting Report Discrepancies

Mission Dates

Some reports use the date from the mission record, some use the date from the mission report. In some cases these are different if the actual date flown is different from the scheduled date. This will only make a difference if the difference carries over from one month to the next.

Mission types

Some reports count only the mission types deemed “statistical” and others count all mission types. The difference is “admin” and “commercial companion” missions.

AFA legs with approved mission reports

Some reports specifically exclude AFA legs, some do not. An AFA leg should never have an approved mission report. If a report for an AFA leg is mistakenly entered and approved, this will cause a discrepancy.

Cancelled missions with approved mission reports

A cancelled mission should never have an approved mission report. If a report is mistakenly entered and approved, this will cause a discrepancy.

Practices to avoid

Once a month has been processed for permanent statistics, any of the following changes to a mission/mission leg record in that month will cause discrepancies:

  • Changing a mission type from or to “admin” or “commercial companion”
  • Changing the approval status of a mission report
  • Changing the cancelled status

If it becomes necessary to correct an error in a mission/mission leg that occurred in a month that has been processed, the permanent statistics should be adjusted accordingly to match. This will need to be done by the database administrator.