Integrated Airline Scheduling
4.5 Evaluation
4.5.1 Comparison
In sections 4.3.4.1 and 4.4.3.1, the best parameter setting for the sequential and simultaneous approach was determined (and the best search strategy for the meta- heuristic, respectively). In addition, each calibrated model was applied to the five test scenarios (see Sect. B in the appendix). The results from these tests are pre- sented in the following to compare both approaches.
Fig. 4.66 presents information on the profit34obtained by each solution approach for each scenario. On average, the simultaneous approach resulted in more profitable schedules for all scenarios than the sequential approach. Even when looking at the individual runs, the best schedule obtained by the sequential approach has a lower profit than the worst schedule from the simultaneous approach.
The higher profit of the schedules from the simultaneous approach might be ex- plained by their higher seat load factors (see Fig. 4.67). As for the profit, not a single run of the sequential approach was able to obtain higher load factors than the worst run of the simultaneous approach.
However, as figures 4.68 and 4.69 show, the seat load factors result from different numbers of passengers and flights. For scenario B and C, the number of flights and the number of passengers were higher for the simultaneous approach than for the sequential approach. For scenarios A and E, the situation is vice versa, however, resulting in the lower seat load factor of the sequential approach. In scenario D, the
34Because of the high penalty costs in the simultaneous approach reducing the fitness value if maintenance restrictions are violated, every solution obtained fulfills this restriction. Thus, the fitness values of the solutions of the simultaneous approach match the profit.
158 4 Integrated Airline Scheduling
Fig. 4.67 Comparison of seat load factors (SLF)
0 0.1 0.2 0.3 0.4 0.5 0.6
A B C D E
SLF
Scenario
Seq. Sim. Seq. Sim. Seq. Sim. Seq. Sim. Seq. Sim.
Maximum Average Minimum Stand. deviation
Fig. 4.68 Comparison of numbers of passengers
0 1000 2000 3000 4000 5000 6000 7000
A B C D E
No. of passengers
Scenario
Seq. Sim. Seq. Sim. Seq. Sim. Seq. Sim. Seq. Sim.
Maximum Average Minimum Stand. deviation
Fig. 4.69 Comparison of numbers of flights
0 20 40 60 80 100 120 140 160
A B C D E
No. of flights
Scenario
Seq. Sim. Seq. Sim. Seq. Sim. Seq. Sim. Seq. Sim.
Maximum Average Minimum Stand. deviation
number of flights was lower for the simultaneous approach. These flights must be more attractive for passengers, because in contrast the total number of passengers is higher than for the sequential approach with its higher number of flights.
Another interesting result when comparing the sequential and simultaneous ap- proach is their standard deviation of the key figures presented (see Table 4.7) on page 123 and Table 4.9 on page 151. In almost every experiment they were lower
Fig. 4.70 Comparison of number of necessary fitness evaluations
0 100000 200000 300000 400000 500000
A B C D E
No. of evaluations
Scenario
Seq. Sim. Seq. Sim. Seq. Sim. Seq. Sim. Seq. Sim.
Maximum Average Minimum Stand. deviation
Fig. 4.71 Comparison of profits of schedules con- structed with the sequential (regular and extended termi- nation criteria) and simulta- neous planning approach
-100000 0 100000 200000 300000 400000 500000 600000 700000 800000
A B C D E
Profit
Scenario
Sim.
Seq. ext. Seq. Sim. Seq. ext. Seq. Sim. Seq. ext. Seq. Sim. Seq. ext. Seq. Sim. Seq. ext. Seq.
Maximum Average Minimum Stand. deviation
for the simultaneous approach. Because the initialization of the GA is conducted randomly with every decision variable (airports and ground times) having the same selection probability, the low standard deviations of the schedules obtained are very encouraging. The – compared to the simultaneous approach – higher standard de- viation of the sequential approach is most likely the result of the combination of deterministic and stochastic search during optimization. The initialization is con- ducted deterministically and the optimization steps perform a greedy – thus, rather deterministic – search, which should result in similar solutions in the end. However, because of the repair mechanisms and the maintenance routing steps, more signifi- cant and stochastic changes are applied to the solutions in the different runs. Because the deterministic search steps are then based on these modified solutions, the search processes take different paths through the search space, resulting in higher standard deviations of the final solutions.
In Fig. 4.70, the number of fitness evaluations that were necessary to obtain the final solution are compared. The GA needs considerably less effort to obtain its solution compared to the traditional approach. Especially for scenarios A and C the GA required only a fraction of evaluations compared to the number of evaluations the sequential approach needed. It has to be emphasized that when comparing both
160 4 Integrated Airline Scheduling
Table 4.10 t-values for the validation of the solution approach comparison
Scenario A B C D E
t-value 6.439 41.899 25.000 21.986 3.708
Fig. 4.72 Comparison of number of necessary fitness evaluations required by the sequential (regular and ex- tended termination criteria) and simultaneous planning approach
0 200000 400000 600000 800000 1e+06 1.2e+06 1.4e+06 1.6e+06 1.8e+06
A B C D E
No. of evaluations
Scenario
Seq. ext. Seq. Sim. Seq. ext. Seq. Sim. Seq. ext. Seq. Sim. Seq. ext. Seq. Sim. Seq. ext. Seq. Sim.
Maximum Average Minimum Stand. deviation
solution approaches the number of iterations is an indicator for the computational effort and complexity. To compare the efficiency of the search in terms of improve- ment steps, the model specification has to be considered. For example, the sequential approach performs a greedy search in each optimization step, thus, many fitness evaluations are conducted before actually applying the optimization step (modi- fication of the current schedule). In contrast, in the GA every fitness evaluation might result in an increase of solution quality – if the randomly applied modification was beneficial. However, these differences result from the underlying optimization techniques and are only of theoretical interest. The number of fitness evaluations necessary determines the effort of the techniques, resulting in the simultaneous ap- proach being the technique that requires less computational effort and time.
To confirm the main results from the comparison – the simultaneous approach generates more profitable solutions with less effort than the sequential approach – a second optimization run is conducted for all scenarios using the sequential ap- proach. To increase the probability of obtaining high quality solutions, its running time is further increased. For each scenario, the optimization run that required the most fitness evaluations is determined. Then, this number is doubled and used as termination criteria for the scenario. For example, the longest run for scenario A re- quired a total of 698,906 fitness evaluations (see Table C.12 in the appendix). Thus, in the following experiments on scenario A the optimization is terminated if the number of fitness evaluations reaches 2·698,906≈1,400,000, suspending imax as termination criteria. For this second set of experiments with the extended termina- tion criteria (ext. Seq.), results of the obtained schedule’s profit and the number of fitness evaluations until the best solution was found are presented in the following figures 4.71 and 4.72. As a comparison, they also include the results from the first experiments of the sequential approach and from the simultaneous approach that
were presented in figures 4.66 and 4.70.35The results clearly indicate that the GA- based simultaneous approach still outperforms the extended sequential approach, although it produced more profitable schedules compared to its first application (at the cost of significantly increased effort).
To validate the results, a standard Student’s t-test is conducted for the results on the profit of both approaches.36The null hypothesis H0is that the observed differ- ences in the profit values are random. Hα says that the differences are a result of the solution approach. The critical t-value for p=0.995 is 3.335, for p=0.999 it is 4.501. The results presented in Table 4.10 for the five scenarios show that the t- values always exceed the critical t-value of the level of significance. Thus, H0can be rejected on the 99.9%-level for all experiments except for scenario D, for which H0
can be rejected on the 99.5%-level. As a consequence, the simultaneous approach significantly produces more profitable schedules than the sequential approach while requiring considerably less effort to obtain these schedules.