Download pdf of this article

An Empirical Study of the Impact of the Expanded National Route Program on Flight Planning and Performance
December 6, 1996

* Cognitive Systems Engineering Laboratory, The Ohio State University, ** Department of Aviation Ohio University, *** NASA Ames Research Center

Summary | In order to improve performance, the ATM system has been evolving in directions that give users more flexibility. Available evidence to date (including results from the studies reported below) indicates that such flexibility has increased efficiency. The theme of this paper, however, is that to further drive the system toward improved performance, users need better information access about air traffic bottlenecks and better feedback about the outcomes of their decisions when planning flights.

In particular, two studies were conducted using data from a major airline to evaluate the impact of the expanded National Route Program (NRP) on fuel consumption. One study compared predicted with actual fuel consumptions for such flights, and found that 35% of the NRP flights routinely burned more fuel than expected. The other studied the flight amendments associated with such flights for the city pair that most often showed fuel losses (LAX-DFW) in order to determine the impact and underlying causes of fuel losses. The observed fuel losses were clearly a result of ATC reroutes to avoid traffic congestion at the northwest cornerpost into Dallas.

More specifically, the following conclusions were reached:
•  Use of the expanded NRP as practiced by this airline is clearly
better than simply filing flights on FAA preferred routes;
•  The actual benefits from such NRP flights are less than predicted, however. In particular, 35% of the flights studied (where a flight is defined in terms of a particular origin, destination, departure time and equipment type) routinely burned more fuel than expected;
•  In the absence of any additional data, it would have been more economical for this airline always to file the FAA preferred route for 7% of the flights studied. The flight from DFW to SNA at 1645 UTC is one example of such a flight. The data indicate that on average this flight burned an additional 1.3% in fuel (over what would have been burned by staying on the FAA preferred route) each time it was filed on an NRP route;
•  If the airline collected data on the performances of flights filed under the expanded NRP, more sophisticated decision rules could be applied to decide when to leave a flight on the FAA preferred route and when to file it under the expanded NRP, resulting in greater fuel savings;
•  Even greater savings would result if the airline could work with the involved TMUs to predict which NRP flights are likely to be rerouted;
•  For the particular city pair studied in detail (LAX-DFW), fuel losses were clearly attributable to traffic congestion problems. It is quite possible for other routes, however, that pilots flying direct instead of staying on the NRP route could be contributing to fuel losses (Interviews with 95 pilots indicated that 45% did not know when they were flying NRP flight plans);
•  This type of analysis could also be used by the FAA for "process control". If information of this type were available from all of the airlines, the FAA would have objective data for identifying the location and costs associated with bottlenecks in the NAS. Such cost data could help guide decisions about whether and how to allocate resources to deal with air traffic bottlenecks.
Conclusions 4-6 all imply the need to develop more effective decision-support tools for dispatchers, as well as for airline and FAA systems analysts.

Finally, this analysis provides a model for how to study the impact of any airline or FAA program that is expected to impact fuel consumption (or flight times), and to identify the underlying causes of problems that arise as part of the implementation of that program. Equally important, it provides objective data on the costs associated with air traffic bottlenecks, thus making it possible to make more informed decisions about resource allocations to deal with such problems.

Background | This report documents two studies dealing with the impact of the expanded National Route Program (NRP) on fuel consumption. Using an analysis of quantitative data on predicted and actual fuel consumptions, along with the results of an observational study comparing actual with filed routes of flight on the Aircraft Situation Display (ASD), we have attempted to develop a more complete picture comparing predicted performance with actual performance.

The motivation for this study came from two sources. First, dispatchers at a number of airlines, as well as traffic managers at enroute air traffic control centers (ARTCCs), have provided numerous examples of how flights filed under the NRP are sometimes given significant amendments, and suggested that some of these changes occur on a regular basis. In some cases, the changes are clearly initiated by the ATC system to deal with traffic congestion. In other cases, the changes are initiated as a result of joint agreement by the flight crew and a controller to amend a flight plan to fly direct.

Along these lines, dispatchers made comments like:

"Under the expanded NRP, it's like shooting ducks in the dark."

"The problem with the expanded NRP is that there's no feedback. Nobody's getting smarter. Someone
has to be responsible for identifying and communicating constraints and bottlenecks."

"It used to be the weather that was the biggest source of uncertainty. Now it's the air traffic system."

As a specific example, one dispatcher indicated that NRP flights from Washington National to Cincinnati frequently have a problem because of the strategy used by ATC to deal with crossing traffic:

"It happens to us all the time. We file the flights at 35 or 39 [altitudes of 35,000 or 39,000 feet] and they're held at 23, 25 and 27. They don't tell us ahead of time that it's going to happen."

A second example of how traffic bottlenecks can affect NRP flights was provided by a traffic manager: "Quite often ... 8-10 extra aircraft are on this northern route to DFW [from Southern California to Dallas flying north of White Sands into the northwest cornerpost at the Dallas-Fort Worth airport] during the noon arrival rush [noon local time]. This causes a sector saturation problem in ZFW Sectors 93 and 47 [two Dallas-Fort Worth (DFW) air traffic control sectors]. To relieve this volume problem, the ZFW TMU [Traffic Management Unit] moves 5 aircraft back to the south route [south of White Sands] via CME.TQA.AQN.DFW [a sequence of navigational fixes into the southwest cornerpost of the Dallas-Fort Worth airport]. This longer route of flight, plus the fact that DFW is in a south flow (meaning these flights will spend more time flying below 10,000 feet), will reduce fuel savings or negate them all together for this bank of flights." (See the figure on the following page.)

Thus, anecdotal evidence suggests that traffic bottlenecks arise that impact the efficiency of NRP flights. Similar anecdotal evidence exists indicating that inefficiencies are introduced when pilots fly direct, rather than following a filed NRP flight plan:

"During the high jet stream season through the middle of the country, one of our dispatchers filed a BWI-LAS late night flight way up to the north, almost in Canada. It was a Boeing 737-300. At departure time the pilot requested a direct to LAS, and received the amended clearance. Later he ended up calling his dispatcher, saying he was short on fuel. He landed short in MCI and ended up being 1 1/2 hours late into LAS."

Most such examples involve less significant, but still important consequences if they occur regularly:

"There was a flight that flew from DFW direct to Parker. I [the dispatcher] had planned it over Albuquerque because of a favorable southerly jetstream. Flying direct to Parker, the flight was flying directly into the jetstream. The plane was 6 minutes late."

In short, interviews with dispatchers and traffic managers suggest that there are sources of inefficiencies that can have an important impact on NRP flights.

Study 1: Analysis of Predicted vs. Actual Fuel Consumption | To look for evidence of such inefficiencies, we collected data from a major airline on all of their flights filed under the NRP from July 1, 1995 to November 30, 1995. These data were used to compare predicted fuel consumption on NRP routes with predicted fuel consumption on FAA preferred routes and also with actual fuel consumption. In the following discussion, a "flight" is defined to be a particular combination of an origin, destination, Ptime (scheduled departure time) and equipment type. Thus a given flight could have a new instance filed each day. Predicted and actual fuel consumptions were from wheels-up to wheels-down.

Comparison of Predicted Fuel Consumptions | Predicted fuel consumptions were first analyzed, comparing performances on FAA preferred routes with the filed NRP routes. 21,334 flight instances were filed by this airline under the NRP during this time period. The average predicted fuel savings per day during this time period ranged from 2.3% to 6.0%. The total predicted savings was 17,723,329 lbs. (See Appendix A for details.)

Comparison of Predicted vs. Actual Fuel Consumption | Given the anecdotal evidence outlined earlier, however, it seems possible that these predictions overestimate actual fuel savings for some flights, since the computer's predictions do not take into account the new reroutings that might occur as a result of filing an NRP route and then encountering a traffic bottleneck while enroute, nor do they consider new cases where the flight crew files a less efficient direct route once airborne. Consequently, we also compared predicted with actual fuel consumption.

To ensure adequate statistical power, only flights with at least 20 instances were considered. There were 267 such flights. A statistical analysis indicated that 94, or 35%, of these flights routinely burned more fuel than predicted (P<0.05). Of these 94, 21% routinely burned more extra fuel than was supposed to be saved by flying the NRP route instead of the FAA preferred route. (The predictions generated 75 minutes prior to scheduled departure were used for this comparison.) The flight from DFW to SNA at 1645 UTC (flying an MD80), for instance, on average burned 1013 lbs. of fuel more than predicted. As a result that flight, which on average was supposed to save 759 lbs. of fuel compared to the FAA preferred route (a predicted 4% savings), actually burned 254 lbs. more than the prediction for the FAA preferred route (a 1.3% loss). Appendix B provides details on all 94 of these problematic flights.

These data also indicated that the city pair that most often had flights with regular problems was LAX to DFW. Seventeen of those flights routinely burned more fuel that predicted.

Study 1. Implications | Minimally, these data indicate that there was some sort of a problem associated with 35% of the flights filed by this airline under the NRP during this time period. One possibility would be an underlying inaccuracy in the prediction model for one or more of these flights, over and above any new problems introduced by use of the expanded NRP. If, however, we assume that the prediction model provides unbiased estimates (after discounting any new problems introduced by use of the expanded NRP), then these data indicate that the actual benefits in terms of fuel consumption from the use of the NRP are less than predicted by this airline.

These results by themselves suggest several directions for action:

•  In the absence of any additional data, it would have been more economical always to file the FAA preferred route for 7% of the flights studied. The flight from DFW to SNA at 1645 UTC (flying an MD80) described above is an example of such a flight, as the data indicate that on average, during the time period observed, this flight burned an additional 1.3% in fuel each time it was filed on an NRP route;
•  An analysis such as the one described above ought to be a routine part of a "process control" program by that airline, looking for "outliers" in NRP flights and then using appropriate resources to identify and deal with the underlying causes;
•  To encourage dispatchers to watch for such problems on a daily basis, control charts could be displayed at the end of each flight instance showing them how it performed relative to expected performance and relative to past performance for that flight;
•  This type of analysis could also be used by the FAA for "process control". If data of this type were available from all of the airlines, the FAA would have objective data for identifying the location and costs associated with bottlenecks in the NAS. Such cost data could help guide decisions about how to allocate resources to deal with air traffic bottlenecks.

Study 2: A Detailed Observational Study of LAX-DFW Flights | As mentioned above, the city pair that most often encountered problems was LAX-DFW. We therefore decided to study it in detail in order to collect more detailed data on the nature of the problems with NRP flights for this city pair, and to better quantify the impact of these problems.

Methods | Four students from the Aviation Department at Ohio University collected data from June 22, 1996 to August 23, 1996 on the performances of flights from LAX-DFW. Flights with five different scheduled departure times (Ptimes) were studied (1400, 1415, 1445, 1515, and 1810 Universal Coordinated Time or UTC). The students collected data on predicted and actual fuel consumptions and observed each flight instance on the ASD to record any flight amendments. They also interviewed pilots immediately after each flight to help document the causes of any flight amendments that arose, and to assess the pilots' understanding of the NRP.

Results | The resultant observations quickly made it clear that the underlying problem was the rerouting described earlier. Very briefly, what happens is:

•  A flight instance is filed under the expanded NRP along a route north of White Sands (special use airspace) to the northwest cornerpost at DFW;
•  While that flight is enroute, the Air Traffic Management (ATM) system decides that there is likely to be a sector saturation problem in the Turkey or Falls high sectors when the flight reaches that point as it approaches the northwest cornerpost into DFW;
•  To deal with that problem, the flight or flights with the most southerly routes that are flying to the northwest cornerpost are rerouted south of White Sands to the FAA preferred route so that they will approach DFW via the southwest cornerpost.

The discussion below provides details on this problem.

Cornerpost Swapping | The table below indicates the frequency with which the cornerpost swap occurred for the different flights that we observed. (Keep in mind that this swap occurs before White Sands, not as the flights are approaching the airport.) The results indicate that the flights that arrive at DFW for the noon rush (flights that are arriving into DFW around noon local time, and that have scheduled departure times or Ptimes of 1400 and 1415 UTC) are particularly affected. 33-39% of the flights during that time period fell into that category and were rerouted south of White Sands to the FAA preferred route.

Table 1. Percentage of Flights Flying the FAA Preferred Route (Pref Route) and NRP Routes with or without Cornerpost Swaps. (Ptime is Universal Coordinated Time or UTC).

Table 2 indicates the impact of this rerouting on overall savings for the NRP flights filed at particular Ptimes. The results indicate that there are significant reductions in overall fuel savings for NRP flights as a result of this cornerpost swapping. For a Ptime of 1400, for example, the computer model predicted a 3.2% average savings in fuel compared to the FAA preferred route, while the real fuel consumption was on average only 0.2% less than that predicted for the FAA preferred route.

Table 3 provides similar data, but only for those instances where an NRP flight was actually rerouted south of White Sands. All of these flights on average burned more fuel than was predicted if they had been filed on the FAA preferred route. On average, for example, it cost an additional 1502 lbs. of fuel each time the flight at 1400 UTC was rerouted to the southwest cornerpost. A statistical test comparing actual with predicted fuels consumptions for these flights was significant (p<.05) for the Ptimes of 1400, 1445 and 1810.

Table 2. Expected vs. Actual Fuel Savings for All Flights Filed Under the NRP (Ptime is UTC; Savings are the % reduction or increase relative to the predicted fuel consumption for the FAA preferred route that day.)

Table 3. Expected vs. Actual Fuel Savings for Those Flights Filed Under the NRP that were Rerouted from the Northwest to the Southwest Cornerpost. (Ptime is UTC; Savings are the % reduction or increase relative to the predicted fuel consumption for the FAA preferred route that day.)

These results make it clear that if it had been possible to predict which NRP flights were going to be swapped to the southwest cornerpost, they should have been filed on the FAA preferred route. Furthermore, even without knowing which NRP flights are going to be swapped, if we knew what the probability was that a swap would occur for an NRP flight, then we could develop more sensitive decision rules for deciding when to leave a flight on the FAA preferred route. The figure on the next page illustrates this approach. Based on the data from this study, it was estimated that there was a 69% chance that an NRP flight leaving LAX at 1400 UTC would be swapped to the southwest cornerpost. Furthermore, it was estimated that on average such flights (DC10s) burned 43431 lbs. of fuel if they were swapped, but only 41929 lbs. if they were left on the NRP route to the northwest cornerpost.

Given these data, the airline would have been better off filing such DC10 flights with a Ptime of 1400 UTC on an NRP route only if they expected to burn more than 42965 lbs. of fuel on the FAA preferred route. This rule would have changed the decision for 22% of the NRP flights filed at 1400 UTC, causing the airline to leave them on the FAA preferred route. (An even more sensitive decision rule would adjust these numbers based on knowledge of the winds for that day.)

Pilot Interviews | 95 pilots from the flights that were observed were interviewed immediately after the conclusion of their flights. Their comments about the causes of the observed reroutes were consistent with the conclusions outlined above. More importantly, the interviews made it clear that 45% of them did not know whether they were on an NRP flight or not, and that many of them had significant misunderstandings about that program, as evidenced by statements like:

"NRP routes are mainly overseas."

"Direct is better than staying on an NRP route."

"The only NRP route from LA goes to Chicago."

While this lack of understanding had little impact on the observed flights from LAX-DFW (given the presence of special use airspace, those flights generally could not be filed direct), such misinformation could be contributing to the problem of pilots accepting clearances to fly direct when they are on an NRP flight plan.

Conclusions | Taken together, these two studies provide strong evidence that, although the available evidence indicates that filing routes under the expanded NRP as practiced by this airline is generally preferable to always filing on FAA preferred routes, the actual savings are likely to be less than predicted (see Appendices A and B and Table 2). In particular, 35% of the flights evaluated in Study 1 were found to routinely burn more fuel than expected when filed under the NRP. Furthermore, the results suggest that, if data were regularly collected on the performances of flights filed under the expanded NRP, more sophisticated decision rules could be developed to decide when to leave a flight on the FAA preferred route and when to file it under the expanded NRP. The results also suggest that even greater fuel savings could result if the airline, working in cooperation with the involved traffic management units, could predict which NRP flights were likely to be affected by traffic congestion.

In the case of the LAX-DFW flights, it is clear that the observed fuel losses were due to the impact of traffic congestion which was not considered by the computer model that generated predicted fuel consumptions for comparisons between the FAA preferred route and alternative NRP routes. For other city pairs that are experiencing problems, it is quite possible that other types of traffic bottlenecks are responsible. It is also possible, however, that some of the losses on these other city pairs are due to pilots accepting direct routes instead of remaining on the filed NRP routes. (This latter problem is not likely to occur often for LAX-DFW flights because of the presence of White Sands between these two cities.) The fact that 45% of the pilots interviewed were unaware of whether they were on an NRP flight plan supports the possibility that this could in fact be a significant contributor to fuel losses.

In general terms, this study illustrates the problems associated with asking people to make decisions based on incomplete information. The expanded NRP shifted the focus of control for selecting flight plans to airline dispatchers, on the assumption that they could make better choices based on their airline's business concerns. There was not, however, a corresponding increase in information access about air traffic bottlenecks. In general, dispatchers have far less information and knowledge about air traffic bottlenecks than did the FAA traffic managers who used to have to approve requests for non-preferred routes. Because of this shift in the focus of control, without a corresponding shift in information access, it appears that dispatchers are in fact to some extent "shooting ducks in the dark." Consequently, there is a great need to develop better information and decision-support systems to assist them with flight planning activities.

Finally, these results provide a model for evaluating other airline and FAA programs, and for providing objective, quantitative data identifying the nature and costs of inefficiencies encountered by these programs. Such results should provide a more objective basis for making decisions about resource allocations to improve the system.

Future Research | We are currently extending this work in three directions. First, we are studying the impact of the new DFW Metroplex plan on the problems documented in this study. Second, we are developing software that will use airline and ASD data to automatically complete the types of analyses conducted in Studies 1 and 2. Third, we are investigating the design of tools to support cooperative decision making between airline operations control centers and FAA traffic management units, so that dispatchers can make more informed decisions when planning flights.

Acknowledgments | This work was supported by the FAA Office of the Chief Scientist and Technical Advisor for Human Factors (AAR-100) and NASA Ames Research Center. We would like to express special appreciation to Larry Cole, Eleana Edens, Tom McCloy, Mark Hoffman, Roger Beatty, Joe Bertapelle, Rob Blume, Scott Ridge, Moira Hoban Edwards and John Tittle.

Appendix A | Predicted Fuel Savings (Total Estsavings is the total estimated savings achieved by filing NRP routes instead of FAA preferred routes for that date; Avg % Estsavings is the % reduction in fuel consumption for the filed NRP routes as compared to the FAA preferred routes)

Appendix B | Predicted vs. Actual Fuel Savings for 94 Flights Where Actual Fuel Consumption Routinely Exceeded Predicted Fuel Consumption (Ptime is UTC; Actual is the mean actual fuel consumption for that flight; Predicted is the mean predicted fuel consumption for that flight; Estsave is the mean estimated fuel savings for that flight or the mean difference between the estimated fuel consumption for the NRP route minus the estimated fuel consumption for the FAA preferred route)