Chapter 9 Results and discussion
9.2 DGA process evaluation
9.2.4 W.A. Lewitt primary school timetabling problem
difference between these two mutation operators when applied to HS1. For data sets HS2, HS4 and HS7, no statistically significant difference is found when comparing the performance of the three operators. From the Z-values for data set HS5, a conclusion is made favouring the use of either the 1V mutation operator or the random swap operator.
The Z-values for data set HS3 allows for the conclusion that the random swap operator performs better than the one violation operator at a 10% level of significance.
Based on the results above, it is concluded that the best mutation operator varies depending on the data set. For the remainder of the runs, the one violation (1V) mutation operator will be used when applying the DGA to data sets HS1, HS2, HS3, HS4 and HS7. The random swap operator will be used when applying the DGA to data set HS5.
Table 9.44: Processes and parameter values to test best construction heuristic (Lewitt problem)
Constant Methods and Operators Phase 1
Selection Variant
Mutation Hybrid mutation
Phase 2
Selection Variant
Mutation Random swap
Constant Parameter Values
SCM 10
Population Size 500
Tournament Size 10
Swaps per mutation 200
Generations 50
The results of using the DGA approach with each of the three construction heuristics are shown in Table 9.45. These are used as secondary heuristics with the consecutive periods heuristic as the primary heuristic.
Table 9.45: Results for best heuristic Success Rates
Saturation Random Largest Degree
40.00% 6.67% 10.00%
Average Hard Constraint Violations (and standard deviations)
Saturation Random Largest Degree
0.73 (0.69) 2.13 (1.2) 2.37 (1.27)
Average Quality (Feasible Timetables)
Saturation Random Largest Degree
12.25 12.00 9.67
Standard deviations (Quality)
Saturation Random Largest Degree
5.38 1.41 3.79
Table 9.45 shows that the DGA using the saturation degree heuristic produces the largest number of feasible timetables. On average, the DGA using the saturation degree produces the fewest number of hard constraint violations. An observation made was that the fitness of the initial populations of the DGA when using the random allocation and largest degree
heuristics were very high. The evolutionary process was then unable to induce feasible timetables.
Hypothesis tests are conducted for feasibility. The hypotheses tested and the Z-values are shown in Table 9.46.
Table 9.46: Hypotheses and Z-values for feasibility
Hypothesis Z-value
H0: µLD = µSD; HA: µLD > µSD 6.07 H0: µRA = µSD; HA: µRA > µSD 5.55
The Z-values in the table above allow for the null hypothesis to be rejected in favour of the alternate hypothesis at all levels of significance. This means that the DGA using the saturation degree heuristic performs better than the DGA using either the random allocation or largest degree heuristics. The DGA using the saturation degree heuristic also produces the best quality timetable with the fewest number of soft constraint violations (7).
9.2.4.2 Comparison of Phase 1 selection methods
The two selection methods are variant tournament selection and standard tournament selection. The constant processes and parameter values used to test the two selection methods are listed in Table 9.47.
Table 9.47: Processes and parameter values to test best selection method (Lewitt problem)
Constant Methods and Operators Phase 1
Heuristic Saturation Degree
Mutation Hybrid mutation
Phase 2
Selection Variant
Mutation Random swap
Constant Parameter Values
SCM 10
Population Size 500
Tournament Size 10
Swaps per mutation 200
Generations 50
The success rate, average quality and standard deviations produced is listed in Table 9.48 and the frequency diagram is shown in Figure 9.12.
Table 9.48: Success rates and average quality for selection methods (Phase 1) Success Rates
Variant Standard
40.00% 43.33%
Average Quality (Feasible timetables)
Variant Standard
12.25 10.54
Standard deviation
Variant Standard
5.38 3.36
Figure 9.12: Frequency chart showing quality for two selection methods
Table 9.48 indicates that the DGA using standard tournament selection produces a slightly higher success rate and better quality timetables than the DGA approach using variant tournament selection. The standard deviation also indicates that the quality of the timetables produced tend to be close to the mean. The frequency chart (Figure 9.12) shows that more quality timetables are produced when using the DGA with standard tournament selection. A hypothesis is tested in order to determine the better selection method. The hypothesis states that the DGA using standard tournament selection performs better than the DGA using variant tournament selection method (in terms of feasibility). A Z-value of 0.17
0 1 2 3 4 5 6 7 8 9
0-9 10-19 20-29 30-39 40-49
Count
No. of Soft Constraint Violations
Frequency Chart for Quality of Timetable
Variant Standard
indicates that there is no statistically significant difference in the performance of the two selection methods.
Standard tournament selection is chosen as the selection method due to its slightly higher success rate in producing feasible timetables. The frequency chart also indicates that DGA using standard tournament selection produces more timetables of better quality than when using variant tournament selection.
9.2.4.3 Comparison of Phase 1 mutation operators
Four mutation operators were considered and the DGA was applied using each of these mutation operators. From all the runs conducted, only one feasible timetable was produced.
The low success rates produced indicate that the mutation operators are not sufficient in producing feasible timetables. The complexity of the problem and the large number of double periods to be allocated contribute to the poor performance of the mutation operators.
In addition, all tuples must be allocated to all available periods. An alternative mutation operator was considered incorporating a combination of 2VH, 1VH and a random swap.
Table 9.49: Processes and parameter values to test best mutation operator (Lewitt problem)
Constant Methods and Operators Phase 1
Heuristic Saturation Degree
Selection Standard
Phase 2
Selection Variant
Mutation Random swap
Constant Parameter Values
SCM 10
Population Size 500
Tournament Size 10
Swaps per mutation 200
Generations 50
The success rates and average hard constraint costs of the DGA using each of the mutation operators as well as the hybrid operator are listed in Table 9.50.
Table 9.50: Success rates using different genetic operators (Phase 1) Success Rates
2VH 1VH 2VNH 1VNH Hybrid
0.00% 3.33% 0.00% 0.00% 43.33%
Average Hard Constraint Cost (and standard deviation)
2VH 1VH 2VNH 1VNH Hybrid
4.57 (1.43) 2.97 (1.07) 16.4 (4.69) 110.67 (13.92) 0.77 (0.82)
The success rates listed in Table 9.50 indicate that the DGA using the hybrid operator is far superior to the DGA using any of the other mutation operators. When using the DGA approach with either the 2VH or 1VH operators, the average number of hard constraint violations ranges from 3 to 5, while the average number of violations when using either the 2VNH or 1VNH operators ranges from 16 (2VNH) to 111 (1VNH). Two factors that contributed to the success of the hybrid operator are the fact that three operators were being applied, thus increasing the probability of finding and removing a constraint violation. This also increased the number of swaps by a factor of three. A third factor is the mix of finding constraint violating tuples and random tuples, thus potentially increasing the probability of moving to a new area of the search space.
The number of feasible timetables produced by the DGA using the hybrid mutation operator was greater than the single feasible timetable produced by the DGA using the 1VH operator.
A hypothesis is tested for feasibility. This hypothesis states that the DGA using the hybrid mutation operator performs better than the DGA using the 1VH operator. A Z-value of 8.97 indicates that at all levels of significance, the performance of the DGA using the hybrid operator is better than the DGA using the 1VH operator.
9.2.4.4 Comparison of Phase 2 selection methods
Phase 2 of the DGA focuses on improving the quality of the timetables. The DGA is used with either variant tournament selection or standard tournament selection. Table 9.51 lists the processes and parameter values used when testing each selection method.
Table 9.51: Processes and parameter values to test best Phase 2 selection method (Lewitt problem)
Constant Methods and Operators Phase 1
Heuristic Saturation Degree
Selection Standard
Mutation Hybrid mutation
Phase 2
Mutation Random swap
Constant Parameter Values
SCM 10
Population Size 500
Tournament Size 10
Swaps per mutation 200
Generations 50
When analyzing the results, no difference in performance is found when comparing the performance of the DGA with the two selection methods. The average quality of the timetables produced is exactly the same. According to the frequency diagram below (Figure 9.13), the DGA using either selection method produces timetables of an equivalent quality.
The DGA using either selection method produces the same number of timetables that have between 0 and 9 (and between 10 and 19) soft constraint violations.
Figure 9.13: Frequency chart for two selection methods
Thus, either selection method can be used for Phase 2. Hypothesis tests are not conducted as the average qualities of the timetables produced when using either method is the same.
9.2.4.5 Comparison of Phase 2 mutation operators
In order to improve the quality of the timetables produced from Phase 1, the DGA uses one of four mutation operators during Phase 2. These operators are the random swap, 1V mutation, 2V mutation and the row swap operators. The row swap mutation operator was immediately not considered since this mutation was found to conflict with the hard constraint regarding double periods. The processes and parameter values used when testing the mutation operators are listed below.
0 1 2 3 4 5 6 7 8 9
0-9 10-19 20-29 30-39 40-49 50-59 60-69
Count
No. of Soft Constraint Violations
Frequency Chart for Quality of Timetable
Variant Standard
Table 9.52: Processes and parameter values to test best Phase 2 mutation operator (Lewitt problem)
Constant Methods and Operators Phase 1
Heuristic Saturation Degree
Selection Standard
Mutation Hybrid mutation
Phase 2
Selection Variant
Constant Parameter Values
SCM 10
Population Size 500
Tournament Size 10
Swaps per mutation 200
Generations 50
The average quality of the DGA when using each mutation operator is listed in the table below (Table 9.53)
Table 9.53: Average quality produced using different soft mutation operators Average Quality
Random Swap 1V 2V
10.54 15.62 24.31
It is evident from the table above that the DGA using the random swap operator produces the best quality timetables. The frequency diagram (Figure 9.14) also illustrates that the random swap produces better quality timetables.
Figure 9.14: Frequency chart showing quality for different soft mutation operators
The DGA using the random swap operator produces timetables that have between 6 and 15 soft constraint violations. Hypothesis tests are conducted for quality and two hypotheses are established. The first hypothesis states that the DGA using the random swap performs better than the DGA using the 1V operator. The second hypothesis states that the DGA using the random swap performs better than the DGA using the two violation (2V) operator.
Z-values of 0.05 and 0.02 respectively indicate that there is no significant difference when choosing between the mutation operators. While not statistically significant, the random swap is the chosen mutation operator to use when addressing soft constraint violations. The DGA using this operator produces better quality timetables on average.