Chapter 9 Results and discussion
9.2 DGA process evaluation
9.2.1 The Abramson benchmark school timetabling problem (HDTT)
9.2.1.1 Comparison of low-level construction heuristics
One of three primary construction heuristics is used to construct the initial population. These heuristics are random allocation, largest degree heuristic and the saturation degree. If two
or more tuples have the same largest degree or saturation degree, then the random allocation heuristic is used as a tie breaker. Table 9.1 lists the other processes and parameter values that were used when testing the three primary construction heuristics.
Table 9.1: Processes and parameter values to test best low-level construction heuristic (HDTT problem)
Constant Methods and Operators
Selection VTS
Mutation 2 Violation Non Hill Climbing (2VNH) Constant Parameter Values
SCM 100
Population Size 1000
Tournament Size 10
Swaps per mutation 20
Generations 50
The success rates and average number of generations taken to find a solution are shown in Table 9.2 below. The average number of constraint violations and standard deviations are listed in Table 9.3. Based on the tables below, the DGA performed best when using the saturation degree heuristic.
Table 9.2: Performance comparison with different construction heuristics
Success Rates Average number of generations
taken to find solution
Random Largest
degree
Saturation
degree Random Largest
degree
Saturation Degree
HDTT4 100.00% 100.00% 100.00% HDTT4 3 2 1
HDTT5 100.00% 96.67% 100.00% HDTT5 9 7 2
HDTT6 43.33% 53.33% 56.67% HDTT6 21 18 7
HDTT7 20.00% 20.00% 20.00% HDTT7 39 30 15
HDTT8 0.00% 3.33% 6.67% HDTT8 50 47 20
In Table 9.2, the success rates for data sets HDTT4 and HDTT5 are very high with feasible timetables being found for almost every run. In addition, the feasible timetables are found within ten generations. The DGA using the saturation degree finds feasible timetables in at most two generations for data sets HDTT4 and HDTT5. These data sets have a relatively small number of resources and are not as constrained as the HDTT6, HDTT7 and HDTT8 data sets. Results using other methods (described in section 9.6.1) also show that feasible timetables were easily induced for these two data sets.
The success rate for the DGA drops as the number of resources increases for the problem (HDTT6, HDTT7 and HDTT8). The averages in Table 9.3 indicate that the DGA using the saturation degree produces timetables with very few constraint violations when compared to the DGA using the other construction heuristics. The saturation degree heuristic contributes to the low averages by producing a fitter initial population than when using the other two construction heuristics. The results listed below are similar to that of other methods used in the literature, thus indicating the difficulty of these data sets.
Table 9.3: Average constraint violations (and standard deviations) found for different construction heuristics
Random Largest Degree Saturation Degree
HDTT4 0 (0) 0 (0) 0 (0)
HDTT5 0 (0) 0.07 (0.37) 0 (0)
HDTT6 1.13 (1.01) 0.93 (1.01) 0.87 (1.01)
HDTT7 1.60 (0.81) 1.60 (0.81) 1.63 (0.85)
HDTT8 15.07 (7.44) 4.07 (3.26) 2.00 (0.64)
A bar chart showing a comparison of the three heuristics is given in Figure 9.1 below. The x- axis represents the data sets that were used and the y-axis represents the success rates.
The success rate is the percentage of runs that have produced feasible timetables. For example in data set HDTT5, the GA approach using the largest degree heuristic produces feasible timetables for 29 of the thirty runs performed. This results in a 96.67% success rate.
Figure 9.1: Comparison of success rates for each heuristic
Each heuristic, when used with the DGA, creates at least one clash free timetable for the data sets HDTT4, HDTT5, HDTT6 and HDTT7. The DGA using the random allocation heuristic does not produce any feasible solutions for the HDTT8 data set. The bar chart illustrates an inverse relationship between the success rate and the number of resources available for each data set. As the number of teachers, classes and venues increase, the data sets become more constrained. Thus, the difficulty in obtaining a solution increases.
Two hypotheses are tested to determine whether the saturation degree is statistically better than the random allocation and largest degree heuristics. The first hypothesis is that the saturation degree heuristic performs better than the random allocation heuristic and the second hypothesis is that the saturation degree performs better than the largest degree heuristic. The hypotheses and the corresponding Z-values are shown in Table 9.4.
Table 9.4: Hypotheses and Z-values for feasibility
Hypothesis Z Values
HDTT4 HDTT5 HDTT6 HDTT7 HDTT8
H0: µRA = µSD; HA: µRD >
µSD
0.00 5.48 1.02 0.16 9.58
H0: µLD = µSD; HA: µLD >
µSD
0.00 0.00 0.26 0.16 3.40
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
120.00%
HDTT4 HDTT5 HDTT6 HDTT7 HDTT8
Percentages
Data sets
Success rates for each construction heuristic
Random Largest Saturation
The table shows that the saturation degree heuristic is significantly better than the random allocation heuristic for data sets HDTT5 and HDTT8. This is significant at all levels (see Table 6.3 in Chapter 6). A Z-value of 3.4 indicates that the saturation degree performs significantly better than the largest degree heuristic for data set HDTT8.
Figure 9.2 shows the average number of generations that the DGA takes to find a solution when applying each of the three heuristics. The x-axis lists the data sets for the problem and the y-axis is the number of generations taken to find a feasible solution.
Figure 9.2: Average number of generations taken per data set
The DGA approach using the saturation degree produces solutions in the fewest number of generations. As was stated earlier, this is due to the ability of the saturation degree to produce an initial population of individuals with good fitness values. The disadvantage of the saturation degree is that the runtime taken to produce the initial population may be slower than the random and largest degree heuristics due to the processing overhead of the saturation degree when recalculating the number of violation free periods for each unallocated tuple. The column chart shows a trend that the GA approach will take more generations to produce feasible timetables if the resources of the problem increase.
The saturation degree heuristic is chosen as the best heuristic for this problem due to its higher success rate and its ability to produce solutions in fewer generations. The low averages and standard deviations listed in Table 9.3 also indicated that this heuristic was the most consistent in terms of producing the least number of constraint violations over the thirty runs conducted.
0 10 20 30 40 50 60
HDTT4 HDTT5 HDTT6 HDTT7 HDTT8
No of generations
Data sets
Average Generations
Random Largest Saturation