InterFAB Expert Talk recommends additional qualitative indices to boost accuracy of STATFOR air traffic forecasts

More than 220 delegates from all over Europe attended the first in a series of InterFAB Expert Talks examining data and performance in Air Traffic Management (ATM) on 2 March 2021. FABEC hosted the presentation entitled: The accuracy of air traffic forecasts, causes and consequences which provided analysis of current Eurocontrol STATFOR forecast methodology by Prof. Dr.-Ing. Hartmut Fricke, Director of the Institute of Logistics and Aviation at the Faculty of Traffic Science, Technische Universität Dresden (TUD). The research identified significant variations amongst individual states between actual and predicted demand, as well as large spreads between the different STATFOR scenarios leading to hundreds of thousands uncertain flights for some regions. This uncertainty can result in excessive costs arising from unused resources or ATC delays if demand was assumed conservative in other states. This has significant consequences for air navigation service providers (ANSPs) and their ability to comply with Single European Sky performance regulations.

Prof. Fricke proposed supplementing STATFOR analyses with additional statistical values such as the Mean Average Percentage Error (MAPE) used in industries with similar heterogenous characteristics and homogenous products to ANSPs. MAPE compares predicted values with actual values to measure the forecast bias and applies qualitative indices to assess forecast accuracy, thereby adding a further technique to the existing STATFOR scores. When TUD tested the MAPE metric on STATFOR results with some slight adjustments to allow distinguishing between over- and underestimations, the research not only provided more precise information, but enabled more qualitative data such as surveillance data and emissions results to be added to the forecast. MAPE is just one example of several techniques available.

“STATFOR high and low scenarios have negative consequences for ANSPs with regard to resource and cost planning. However, there is an opportunity to introduce other quality indicators,” said Prof. Fricke. “For example, the Enhanced Trajectory Assessment System (ETAS) estimates emissions for thousands of overnight flights in German airspace since 2016 using standard aircraft flight plan data, weather data, and engine combinations verified against the ICAO emissions database. The result would be an enriched STATFOR forecast incorporating real data and providing a platform with strong environmental credentials to support air traffic recovery following the pandemic.”

In a positive exchange of views, representatives from across the industry shared in the discussion. The controllers’ union GATCO observed basing financial decisions simply around forecasts presents many problems and called for a more balanced and sustainable approach, using forecast as a support tool rather than the basis for decision-making. STATFOR welcomed the analysis and added the forecast already provides tables showing MAPE values, while environmental specialists within Eurocontrol produce a 20-year CO2 forecast. However, the real challenge lies in summarising and prioritising the different values to deliver reliable results. Commentators also raised questions around the impact of unpredictable events and political decisions on forecast accuracy.

You can view the recording of the session here: https://www.fabec.eu/registrations/experttalks/