Dynamic Strategic Planning
4.3 The Forecast Is “Always Wrong”
services and public consumption of air travel. Political changes, such as the collapse of the Soviet Union and the dismantling of the traffic barriers between Russia, China, and the West, vastly reconfigured traffic patterns. The list of reasons why trends do not continue over a reasonable planning period is practically endless.
Moreover, asChap. 19on forecasting indicates, even to the extent that trends do contin-ue, the mathematical methods for determining them are too subjective to permit analysts to determine definitively what that trend might be. In short, better methods or better analysts will not make forecasts more reliable. In fact, in the increasingly deregulated world of air transport, forecasts are likely to become even more unreliable than they have been.
The presentation of the track record for aviation forecasts first covers two substantive contexts: the estimation of costs and the forecasts of overall levels of traffic. It then indic-ates how longer planning periods and deregulation of aviation further increase the lack of reliability of forecasts. The object is to provide a sense of the large range of uncertainty that should be attached to any aviation forecast—and thus to all planning scenarios.
Cost Estimation
Estimates of construction costs for major projects are notoriously inaccurate. Differences between estimated and actual costs of 30 percent are common on standard projects, because of surprises on site, changed orders from the architects or owner, and the whole litany of things that can go wrong. On innovative, high-technology projects, these differences can be much larger, and analysts of project costs have observed standard deviation equal to the estimate. Benz (1993) provides an account of what has been observed in all kinds of fields of construction and production, andFig. 4.1summarizes those findings. Notice that he re-ports an overall standard deviation of about 40 percent. In round numbers this implies that, in one case out of three, actual costs differ from estimated costs by more than plus or minus 40 percent.
FIGURE 4.1 Average ratios of actual to estimated costs in various areas. (Source: Benz, 1993.)
An analysis of the cost of resurfacing airport runways illustrates the range of uncertain-ties in cost estimation. As this particular job is about the simplest to estimate, it provides a conservative indication of the uncertainties to be expected. The process of resurfacing run-ways uses primitive technology (asphalt is dumped off trucks and rolled to grade) on a clear surface with no hidden surprises.Figure 4.2illustrates the distribution of the ratio of actual to estimated costs using data from the FAA Western Region of the United States (Knudsen, 1976). The analyst properly adjusted these data for inflation in the cost of construction over the time between the estimate and the execution of the job. Not surprisingly, the average and median actual costs are higher than estimated (about 25 percent in this case). What is remarkable is that the range of costs can be twice or half the average cost!
FIGURE 4.2 Distribution of ratio of actual to estimated costs for runway resurfacing pro-jects. (Source: Knudsen, 1976.)
A general explanation for why actual costs vary from estimates has to do with the fluc-tuations in the real cost of materials and labor. Construction processes use commodities, such as steel and cement, whose production consumes a lot of energy. Thus their prices, and those of petroleum derivatives such as asphalt, rise and fall with the cost of petroleum, which is highly volatile: from 1999 through 2012 the price of a barrel of oil fluctuated between a low of about $10/barrel to over $130. Moreover, the variations in the price of oil, and the effective price of labor, depend largely on the state of the economy (de Neufville et al., 1977; de Neufville and King, 1991). During boom periods, the supplies of oil and labor are tight, so oil prices are high and employers have to pay overtime and premium wages.
During recessions, however, supplies are plentiful, oil prices tend to drop, and workers are less demanding.
Aggregate Forecasts
The periodic swings in the overall economy naturally affect the overall level of aviation traffic. In boom periods, businesses need to travel and individuals have the money to do so. When there is a global or economic crisis, the growth in airport traffic correspondingly slows or decreases. The phenomenon makes medium-term forecasts, those covering 5 to 10 years, distinctly unreliable.
As a rule of thumb, half the medium-term forecasts differ from the forecast by more than 20 percent. This approximation is validated by repeated comparisons between fore-casts and actual results over long periods and in different countries. For over 50 years for
example, the U.S. FAA has each year been preparing both national forecasts of traffic and airport operations over the following 5 years—and reporting the actual levels observed for all the same categories (BTS, Annual). The comparison of such series, in the United States and elsewhere, demonstrates that 5- and 10-year forecasts are indeed easily wrong by more than plus or minus 20 percent.
Anyone can document the phenomenon by making similar comparisons. Figure 4.3 shows one such comparison. It shows how actual results easily deviate from forecasts in just a few years. Although many forecasts were off by over 30 percent, the data in Fig.
4.3are especially conservative: the analysis done in 2004 deliberately omitted results for 2001 and 2002 when terrorism attacks and an economic downturn held back traffic growth.
Those years were indeed extraordinary, but extraordinary events keep happening! If those years had been included, the deviations between forecasts and actual results would be even more striking.
FIGURE4.3 Distribution of ratio of actual to forecast passenger traffic based on FAA Ter-minal Area Forecasts done in 1992–1995 for the years 1997–2000. (Source: Friedman, 2004.)
Table 4.1shows similar information from Japan, taken from their periodic forecasts of their national 5-year investment plans and subsequent statistics on passengers. For Japan, the average discrepancy between the forecast and the actual number of international pas-sengers between 1980 and 1995 was 22 percent after 5 years and 40 percent after 10 years (Nishimura, 1999).
Source: Nishimura, 1999, from Japanese Ministry of Justice Embarkation and Disembarkation Statistics, and Ministry of Transportation, Airport Investment 5-Year Plans.
TABLE4.1 Comparison of 10-year Forecasts of International Passengers to Japan with Ac-tual Results
Table 4.2gives further insight into the inaccuracy of forecasts. It shows long-range fore-casts for Sydney, Australia, prepared by three different authoritative groups. One came from a reputed international consultant, a second was the official forecast of the previ-ous planning study, and the third was the forecast of the Australian Ministry of Aviation.
Although these experts were all working with similar data, their estimates of the future differed widely. None of their forecasts was close to the actual level of traffic some 20 years later.
Sources: Australia Department of Aviation, 1985; Sydney Airport, 2001.
TABLE4.2 Comparison of Actual and Forecast International Passengers through Sydney
Table 4.2 provides further lessons about forecasting in practice. Readers should notice the third- and fourth-place accuracy of the forecasts. This kind of precision is wholly un-justified. Most 20-year forecasts for aviation are lucky to get the first two decimal places right. Reporting more decimal places is pretentious. Two of the forecasts for Sydney have the great merit of providing ranges, reinforcing the notion that forecasts are not precise.
However, these ranges are much too tight. They provide a range of only about plus or minus 20 percent over 20 years. By contrast, as the experience in the United States indicates, we can expect such deviations in a few as 5 or 10 years. The lessons are that aviation planners should do the following:
• Focus on the first two decimal points
• Use large ranges, on the order of plus or minus 30 percent or more over 20 years Simply stated: large forecasting errors are normal.
Effect of Longer Planning Periods
The discrepancy between forecast and reality increases for longer forecasts. This is entirely to be expected. In the short run, inertia in the system keeps things moving as they were. In the longer run, trend-breaking events are more likely. Entirely new travel patterns may set in and make the actual results differ much more from forecasts.
A comprehensive analysis of airport master plans demonstrated the increase in forecast errors for longer-term predictions. Table 4.3 clearly shows how all measures of the error become larger with longer-term forecasts: the average discrepancy, the absolute range of the error, and the consequence standard deviation of the error.
Source: Maldonaldo, 1990.
TABLE 4.3 Discrepancies between the Forecast and the Actual Results Increase for Longer-Term Forecasts
The discrepancies between the forecasts and actual results apply equally to the content of master plans. By looking at old master plans and comparing them with what actually is built, it is easy to calculate statistics similar to those reported so far. In doing this exercise, the analyst has to consider both the projects in the plan that were not built and those that were built that were not in the original plan.Table 4.4summarizes the results for one such study. It shows that the master plans accounted for less than half the projects constructed.
As should be expected, the average discrepancy becomes larger for longer planning hori-zons.
Source: Maldonaldo, 1990.
TABLE4.4 Discrepancies between Projects Forecasted in Master Plans and Actually Built Increase for Longer-Term Forecasts
Effect of Economic Deregulation
Economic deregulation increases the volatility of traffic. This is because deregulation re-moves the barriers to changes in prices, frequency of service, and routes (de Neufville and Barber, 1991). Airlines can and do make sudden major changes in these circumstances, and may radically disturb the patterns and levels of traffic. These moves may have substantial effects on the largest airports. At smaller airports they may cause traffic to double or halve in just a few years. For example, Continental introduced a low-fare service to Greensboro, North Carolina, and doubled traffic from 2 to 4 million total passengers between 1993 and 1995. By 1997 Continental terminated this service, and traffic at Greensboro had fallen back to about 2 million passengers a year. Similarly, in the 1990s Delta created a con-necting hub Cincinnati and built traffic up to a traffic peak of 22.8 million passengers in 2005—and then shut down these operations so that by 2010 traffic at Cincinnati had fallen below 8 million passengers.
Low-cost airlines such as Southwest or Ryanair can suddenly arrive on a market and gen-erate huge increases in traffic. These may persist, or may fall if the airline fails. Southwest has continued to be successful over a generation, whereas PEOPLExpress rapidly doubled
traffic at New York/Newark in the 1980s, from about 10 to about 20 million before it went bankrupt and deflated the traffic by half.
Major airlines can likewise make substantial moves. American Airlines moved a sub-stantial block of its traffic from Chicago/O’Hare to Dallas/Fort Worth at the time deregula-tion became effective in the United States, thus dropping traffic through Chicago by about 15 percent in 1 year asFig. 4.4shows.
FIGURE4.4 Volatility of air traffic at Chicago/O’Hare after deregulation in 1978.
Such radical changes in traffic can obviously affect the performance of an airport drastic-ally. When US Airways moved the focus of its international operations from Baltimore to Philadelphia, the Baltimore airport was left with an underutilized international passenger building. Meanwhile, Southwest Airlines was expanding, so the total traffic at Baltimore stayed steady. However, Baltimore meanwhile had to build new facilities to accommodate this other form of traffic.Chapter 14describes this case in detail.
The bottom line for airport planners and operators is that traffic can change rapidly in a deregulated environment. As of 2012, deregulation was the standard context in the busiest international markets: North America, Europe, Japan, India, Australia, and elsewhere. As this pattern spreads to other free-trade areas, and through “open-skies” policies allowing airlines to serve destinations in other countries freely, this volatility becomes increasingly important.
The fact that the forecast is “always wrong” means that master plans built around spe-cific forecasts will also be “always wrong.” This means that to get the planning right, it is necessary to move away from the notion of planning around a fixed forecast.