Development Of System Safety Performance Measures In Support Of The Global Analysis And Information Network

In May 1996, the Federal Aviation Administration (FAA) announced a new and innovative approach to reach the goal of "zero accidents," known as the Global Analysis and Information Network (GAIN). This is envisaged as a privately owned and operated international information infrastructure for the collection, analysis, and dissemination of aviation safety information that would involve the use of a broad variety of worldwide aviation data sources, coupled with comprehensive analytical techniques, to facilitate the identification of existing and emerging aviation safety problems. In support of this effort, the objective of the research project described in this paper is to assist the FAA in developing aviation system safety performance measures that can utilize the automated operational data that is available in the aviation system, such as digital flight recorder data and air traffic control (ATC) system data, to monitor trends in the operation of the aviation system and anticipate problems before they lead to incidents and accidents. For this to be done, it will be necessary to develop effective techniques to manage the vast amounts of data involved and appropriate analytical techniques to sort through these data and apply formal models to identify situations of interest. The goal of the current phase of the research is to review the available data sources and the techniques that have already been developed by the airlines and FAA, in order to provide the FAA Office of System Safety with a roadmap of what could be done to utilize these data sources to develop safety performance measures and what additional resources this would require. During the course of the research, discussions were held with some 25 people representing a broad range of FAA offices and industry organizations, including the FAA Office of System Safety, Office of Aviation Research, Air Traffic Service, Flight Standards Service, Office of System Capacity and the William J. Hughes Technical Center. Site visits were made to gather information on the FAA Airport Movement Area Safety System, and the NASA Aviation Performance Measuring System and Surface Movement Advisor program. Information was assembled on existing sources of operational data and data analysis tools, including those developed to support Flight Operations Quality Assurance programs. Development of the type of system safety performance measures discussed in this concept paper offers the potential to provide three useful contributions to improving aviation safety. The first is to provide a means to monitor progress toward achieving the FAA goal of reducing the fatal aviation accident rate by 80% by 2007. Since aviation accidents, particularly for commercial airline operations, are relatively rare events, it will take many years of data before any change in the underlying accident rate can be established with any confidence, much less the effect of any particular set of measures to improve the level of safety. Therefore, what are needed are performance measures that are responsive to procedural and technology changes, and measure events that occur much more frequently but reflect situations that those operating the system agree they wish to avoid, as well as conditions that aviation safety experts agree could be indicators of potentially hazardous situations. The second contribution is to provide a means for managers and supervisors to assess the effectiveness of operational changes, to identify locations or procedures that appear to generate a high number of undesired situations, and to tailor the training of flight crews and controllers to help them improve their performance. The third contribution is to provide an early warning indicator of problems that may be emerging from the introduction of new technology, new procedures, and the growth in traffic levels. For this to be achieved, it will be necessary to develop the appropriate performance measures in close collaboration with those involved in operating the system on a day-to-day basis, and to encourage a thoughtful debate on how to measure safety within the NAS and how to improve it. The operational aspects of computing appropriate performance measures are likely to be at least as difficult as deciding what to measure. Fortunately, there already exists considerable experience working with aircraft flight recorder data, and specialized analysis tools have been developed and continue to be developed, by both commercial vendors and government agencies. There is also considerable experience within the FAA and other organizations in working with radar flight track data. Experience in applying these and similar techniques to the development of system safety performance measures will determine what is useful, and whether the effort involved is justified by the results. Therefore it is proposed that a limited number of proof of concept studies should be undertaken as soon as possible to gain experience in developing appropriate analysis tools and techniques, as well as to begin involving the operating community in the process. One such study has already been proposed by researchers at NASA Ames Research Center to explore the application of concepts and techniques developed under the NASA Aviation Performance Measuring System to air traffic control system data. This has been jointly funded by the NASA Aviation Safety Program and the FAA, and is about to commence. The study would utilize the Performance Data Analysis and Reporting System, that is currently under development, to examine routine operational data in order to identify exceedances from normal operations. While the development of a comprehensive approach to measuring system safety performance needs to integrate all available information, including that derived from monitoring aircraft flight operations, as well as the operation of the ATC system, there are immediate opportunities to identify and track safety performance measures using ATC system data. Developing and implementing these measures can not only provide near-term indicators of system performance, but in the longer term can provide a consistent data stream that can be integrated into a more comprehensive framework as the other elements of this framework are implemented. It is recommended that at least two other studies be initiated addressing this aspect of the system, one focusing on terminal airspace operations and the other on airport surface operations. The first study would utilize existing tools, such as those being developed under the NASA Aviation Safety Program, to analyze radar track and system message data for a Terminal Radar Approach Control facility to identify situations that represent a departure from normal operations, including atypical controller intervention to maintain separation, unstabilized approaches, and Traffic Alert and Collision Avoidance (TCAS) alerts. The second study would explore how to effectively utilize the available sources of data on aircraft movement on the airport surface to implement safety performance measures, and would develop algorithms for extracting and analyzing data on the aircraft paths on the taxiway and runway system. The scope and level of effort of these studies could be tailored to the available resources, but it is envisaged that each of these studies would last between six months and a year, and would require between one and two person-years of effort.