Experimental Results

The enhanced Best Performance Algorithm for the Annual Crop Planning Problem Based on Economic Factors

3.9 Experimental Results

The dataset used for this experiment is the dataset listed in Table 3.2 under section 3.7.1. This dataset relates to the VIS. Table 3.4 gives the lower and upper bound settings, the fixed costs of production (𝐹_𝑘𝑖𝑗), as well as the demand equations used for the experiment. For the purpose of simulation, demand equations were formulated for each crop using the statistics of the equilibrium price ton^-1 of yield (i.e. the 𝑀𝑃_𝑘𝑖𝑗), and the hectares allocated (i.e. the 𝐻_𝑘𝑖𝑗).

The parameter settings of metaheuristic algorithms influence their performance per problem instance.

Therefore, for fair algorithmic comparisons for this problem instance, experiments will be performed to determine the appropriate parameter settings for each metaheuristic algorithm. Determining the parameter settings will be the first set of experiments. Once the parameter setting for the algorithms have been determined, the second set of experiments will be performed for the algorithmic

69

comparisons. The 𝑤𝑜𝑟𝑘𝑖𝑛𝑔 solution at each iteration, for all experiments will be determined as follows: randomly select a crop, and thereafter randomly select its hectare allocation.

Table 3.4: Parameter settings per crop Crops 𝑳𝒃𝒌𝒊𝒋 𝑼𝒃𝒌𝒊𝒋

𝑭_𝒌𝒊𝒋 (ZAR)

𝑬𝑷_𝒌𝒊𝒋 (Demand Eq.)

Pecan Nuts (p) 50 300 875,000 30*X + 500

Wine Grapes (p) 100 500 2,864,250 5*X + 510

Olives (p) 100 800 2,700,000 7*X – 300

Lucerne (p) 7,000 8,000 948,416 (2/5)*X + 1814.48

Cotton (s) 1,000 3,000 393,750 2*X + 500

Maize (s) 5,000 8,000 8,323,875 X/4 - 303.75 Groundnuts (s) 4,500 9,500 1,522,800 X/2 + 1576

Barley (w) 100 300 7,249,779.6 10*X + 83.27 Wheat (w) 10,000 15,000 1,565,740.8 X/6 + 174.64

For problem instances where the optimal solution is known, the objective in comparing algorithmic performances is to monitor which algorithm will determine the optimal solution in the shortest computational time. Therefore, with this being the intent, the parameter settings would need to be adjusted accordingly. Another alternative, in comparing algorithmic performances, is to run simulations for a fixed number of iterations. With this approach, the parameter settings would need to be adjusted to make the most effective use of the limited computational time available. One possible problem with this approach is that if the metaheuristic algorithm shows a clear convergence, in leading towards its best solution, this strategy would be ineffective if the termination were to be done before this point of convergence. Therefore, for these reasons, the stopping criterion adopted in this study is to execute the termination of the algorithms at their points of convergence.

Convergence is the point where further improvements in the solution quality would yield minimal benefits compared to the relatively large number of iterations required to yield those minimal benefits.

Therefore, in this study, convergence will be detected when no further improved best solution is found for a large number of iterations. For the experiments to determine the parameter settings, a total of 30,000 idle iterations will be used to detect convergence. Thereafter, in comparing algorithmic performances, a total of 50,000 idle iterations will be used to detect convergence.

70

The experiments run to determine the parameter settings for the probability factor (𝑝_𝑎) and the 𝑃𝐿 size of the eBPA and the BPA can be seen in Figures 3.3 to 3.9 below. In Figure 3.3 and 3.7, the 𝑃𝐿 size remained fixed at 50, while the 𝑝_𝑎 values were randomly selected from within the range of 0 <

𝑝_𝑎≤ 0.15 for the eBPA, and 0 < 𝑝_𝑎≤ 0.25 for the BPA. This was per run for a total of 100 runs per experiment, in using the same initial solutions. Figure 3.4 is a zoomed in image of Figure 3.3, and Figure 3.8 is a zoomed in image of Figure 3.7. The zoomed in images show more clearly the best solutions determined.

Figure 3.3 and Figure 3.7 show that with probability factors below 0.0781 and 0.886 respectively, many solutions were determined which were found in regions that were far away from those of the best solutions found. However, it is seen that in both of these figures that there are no distinguished best values for the 𝑝_𝑎 values, as competitive solutions can be seen scattered throughout the probability ranges. This shows that irrespective of the values of the 𝑝_𝑎’s, the eBPA and the BPA would find good neighborhood regions with more consistency if the probability factors were to be greater than 0.077 and 0.885 respectively. The best solution determined for the eBPA, as seen in Figure 3.4, had a probability factor of 0.128 (truncated to three decimal places). The best solution determined for the BPA, as seen in Figure 3.8, had a probability factor of 0.121 (truncated to three decimal places).

Therefore, for the rest of the experiments, the probability value of 𝑝_𝑎= 0.128 will be used for the eBPA, and the probability value of 𝑝_𝑎= 0.121 will be used for the BPA.

For the experiments run to determine the 𝑃𝐿 size’s of the eBPA and the BPA, the probability value of 𝑝_𝑎= 0.128 remained constant for the eBPA, and the probability value of 𝑝_𝑎= 0.121 remainded constant for the BPA. The values of the 𝑃𝐿 size’s were then randomly selected from within the range of 1 ≤ 𝑃𝐿_𝑠𝑖𝑧𝑒 ≤ 200 for each algorithm per experiment. Again, this was per run for a total of 100 runs per experiment, in using the same initial solutions. For the eBPA, the results are seen in Figures 3.5 and 3.6. For the BPA, the results are seen in Figure 3.9. Figure 3.6 is a zoomed in image of Figure 3.5.

From Figures 3.5 and 3.6, it is seen that the most consistent performances were determined in using 𝑃𝐿 sizes within the range of 18 to 112 for the eBPA. From Figure 3.9, it is seen that the most consistent performances were determined using 𝑃𝐿 sizes greater than 132. However, it is again observed that the eBPA and the BPA determined competitive solutions throughout the 𝑃𝐿 size ranges. For the

71

eBPA, the best solution had a 𝑃𝐿 size of 69; this value will be used for the algorithmic performance comparison tests. For the BPA, the best solution had a 𝑃𝐿 size of 164; this value will be used for the algorithmic performance comparison tests.

With the termination criterion to be set at 𝑥 (i.e. either 30,000 or 50,000) idle iterations, the strategy to be used to reduce of the 𝑃𝐿 size for the eBPA, until a size of 1 is reached, will be as follows: If half of the termination number of idle iterations have been reached (i.e. 𝑚𝑖𝑛𝑖𝑚𝑢𝑚_𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 = 𝑡𝑒𝑟𝑚𝑖𝑛𝑎𝑡𝑖𝑜𝑛_𝑐𝑟𝑖𝑡𝑒𝑟𝑖𝑜𝑛/2), divide the remaining number of iterations by the current 𝑃𝐿 size (i.e. 𝑟𝑒𝑑𝑢𝑐𝑡𝑖𝑜𝑛_𝐶𝑟𝑖𝑡𝑒𝑟𝑖𝑜𝑛 = (𝑡𝑒𝑟𝑚𝑖𝑛𝑎𝑡𝑖𝑜𝑛_𝑐𝑟𝑖𝑡𝑒𝑟𝑖𝑜𝑛 − 𝑚𝑖𝑛𝑖𝑚𝑢𝑚_𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛)/𝑃𝐿_𝑠𝑖𝑧𝑒). If the lower bound plus the reduction criterion (i.e. 𝑚𝑖𝑛𝑖𝑚𝑢𝑚_𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 + 𝑟𝑒𝑑𝑢𝑐𝑡𝑖𝑜𝑛_𝐶𝑟𝑖𝑡𝑒𝑟𝑖𝑜𝑛) equates to the current number of idle iterations then reduce the 𝑃𝐿 size by 1. The reduction of the 𝑃𝐿 has the dual purpose of increasing exploitation, as well as eliminating the possibilities of cycling for 𝑃𝐿 sizes greater that one.

Figure 3.3: Fitness values determined using randomly selected probability factors at a fixed PL size of 50

Figure 3.4: Zoomed in image of Figure 3.3

0.077090105, 336788740

335500000 336000000 336500000 337000000 337500000 338000000 338500000

0 0.05 0.1 0.15

Fitness Value (ZAR)

Probability

eBPA: Fitness Values of Variable Probability at a Fixed Performance List Size

PL_s𝑖𝑧𝑒= 50

0.128687418, 338347584

338260000 338270000 338280000 338290000 338300000 338310000 338320000 338330000 338340000 338350000 338360000

0 0.05 0.1 0.15

Fitness Value (ZAR)

Probability

eBPA: Fitness Values of Variable Probability at a Fixed Performance List Size (zoomed)

PL_s𝑖𝑧𝑒= 50

113, 336794815

336600000 336800000 337000000 337200000 337400000 337600000 337800000 338000000 338200000 338400000 338600000

0 50 100 150 200

Fitness Value (ZAR)

Performance List Size

eBPA: Fitness Values of Variable Performance List Sizes at Fixed Probability _{𝑝𝑎= 0.128}

17, 338211637

69, 338350015

338140000 338190000 338240000 338290000 338340000 338390000

0 50 100 150 200

Fitness Value (ZAR)

Performance List Size

eBPA: Fitness Values of Variable Performance List Sizes at Fixed Probability (zoomed) 𝑝𝑎= 0.128

72

Figure 3.5: Fitness values determined using randomly selected PL sizes at a fixed probability factor of 0.128

Figure 3.6: Zoomed in image of Figure 3.5

Figure 3.7: Fitness values determined using randomly selected probability factors at a fixed PL size of 50

Figure 3.8: Zoomed in image of Figure 3.7

Figure 3.9: Fitness values determined using randomly selected PL sizes at a fixed probability factor of 0.121

The experiments run to determine the parameter settings for SA are seen in Figures 3.10 and 3.11 below. In Figure 3.10, the initial temperature 𝑇 was fixed at 100, while the cooling factor 𝛼 had been randomly selected from within the range of 0.95 ≤ 𝛼 < 1. This was done per run for a total of 100 runs in using the same initial solution. The cooling factor 𝛼 controls the rate of convergence, and decreases 𝑇 using the equation 𝑇 = 𝑇 ∗ 𝛼. Therefore, the higher the value of 𝛼, the slower the rate of convergence, and the more successful the annealing process will be. From Figure 3.10, it is observed that the fitness qualities of the solutions were similar in having found similar neighborhood regions. The best value of 𝛼 seen is 0.96 (rounded off to two decimal places).

0.088578039, 336824492

336500000 336700000 336900000 337100000 337300000 337500000 337700000 337900000 338100000 338300000 338500000

0 0.05 0.1 0.15 0.2 0.25

Fitness Value (ZAR)

Probability

BPA: Fitness Values of Variable Probability at a Fixed

Performance List Size 𝑙𝑖𝑠𝑡𝑆𝑖𝑧𝑒= 50 0.121826845,

338352926

338310000 338315000 338320000 338325000 338330000 338335000 338340000 338345000 338350000 338355000 338360000

0 0.05 0.1 0.15 0.2 0.25

Fitness Value (ZAR)

Probability

BPA: Fitness Values of Variable Probability at a Fixed Performance List Size ^{𝑙𝑖𝑠𝑡𝑆𝑖𝑧𝑒= 50}

164, 338352973

132, 338327097

338305000 338310000 338315000 338320000 338325000 338330000 338335000 338340000 338345000 338350000 338355000 338360000

0 50 100 150 200

Fitness Value (ZAR)

Performance List Size

BPA: Fitness Values of Variable Performance List Sizes at Fixed Probability ^𝑝𝑎= 0.121

73

The value of 𝛼 = 0.96 remained fixed for the experiment related to Figure 3.11. In this experiment, the initial temperature 𝑇 was randomly selected from within the range of 1 ≤ 𝑇 ≤ 500. This was done per run for a total of 250 runs in using the same initial solution. More runs were needed to determine 𝑇, as 𝑇 importantly controls the transition from exploration to exploitation. The parameter settings for SA are more difficult to determine, and would explain the volume of research done on SA. From Figure 3.11, it is seen that the best solution for 𝑇 was 226. Together with 𝛼 = 0.96, these will be the parameter settings to be used for SA in performing the algorithmic comparison tests.

Figure 3.10: Fitness values determined using randomly selected cooling factors, at a fixed initial temperature of 50

Figure 3.11: Fitness values determined using randomly selected initial temperature values, at a fixed cooling factor of

0.96

The experiments run to determine the 𝐶𝐿 size for TS is seen in Figures 3.12 and 3.13. Figure 3.13 is a zoomed in image of Figure 3.12. For this experiment, a recommended 𝑇𝐿 size of 7 was used (Glover, 1986). 𝐶𝐿 sizes were randomly selected from within the range of 1 ≤ 𝐶𝐿_𝑠𝑖𝑧𝑒 ≤ 500. This was done per run for a total of 100 runs in using the same initial solution.

Figure 3.12 shows that 𝐶𝐿 sizes above 209 determined solutions that had fitness values which were far from the best solution found. The best solution found, as seen more closely in Figure 3.13, had a 𝐶𝐿_𝑠𝑖𝑧𝑒 of 34. Figure 3.13 also shows a cluster of competitive solutions found around the 𝐶𝐿_𝑠𝑖𝑧𝑒 of 34. This indicates that a size of 34 is a good value to choose. These values are the parameter settings that will be used for the TS in performing the algorithmic comparison tests.

0.960611544, 330918255

320000000 322000000 324000000 326000000 328000000 330000000 332000000

0.95 0.96 0.97 0.98 0.99 1

Fitness Values (ZAR)

Cooling Factor

Fitness Values of Variable Cooling Factors at a Fixed Initial

Temperature _{𝑇= 100}

226, 332777168

320000000 322000000 324000000 326000000 328000000 330000000 332000000 334000000

0 100 200 300 400 500

Fitness Values (ZAR)

Initial Temperature

Fitness Values of Variable Initial Temperatures at a Fixed

Cooling Factor _{𝛼= 0.96}

74

Figure 3.12: Fitness values determined by randomly selecting the CL size values

Figure 3.13: Zoomed in image of Figure 3.12

As can be seen from Figures 3.3, 3.5, 3.7 and 3.9 the parameter settings for the eBPA and the BPA did not significantly hinder its performances. This is an interesting observation in being compared to an algorithm such as SA which requires more effort to set its parameter values.

For the second experiment, in comparing the algorithmic performances, the parameter settings determined from the first set of experiments were used. For this experiment, a total of 50 runs per metaheuristic algorithm were executed. The termination criterion was 50,000 idle iterations. For each of the 50 runs, per algorithm, the same initial randomly generated solution was passed in as an input parameter to each algorithm. The experiments performed, together with these test criterion, were sufficient to ensure fair algorithmic comparison tests. From the 50 solutions determined by each algorithm, their overall best and average solutions are documented. Their 95% Confidence Interval⁴ values are also documented for their fitness values.

Table 3.5: Average execution time performances (AVG) in milliseconds (ms)

Methods AVG (ms)

BPA 218,093

eBPA 148,178

TS 52,367

SA 33,029

4 The Confidence Interval (CI) indicates the reliability of an interval estimate of population parameters. 95%

CI means to be 95% certain that the population parameters will lie within the interval estimate range.

210, 318741732 34, 338327841

315000000 320000000 325000000 330000000 335000000 340000000

0 100 200 300 400 500

Fitness Value (ZAR)

Candidate List Size

Fitness Values of Variable Candidate List Sizes

𝑇𝐿_𝑠𝑖𝑧𝑒= 7 34, 338327841

335500000 336000000 336500000 337000000 337500000 338000000 338500000

0 100 200 300 400 500

Fitness Value (ZAR)

Candidate List Size

Fitness Values of Variable Candidate List Sizes (zoomed) 𝑇𝐿_𝑠𝑖𝑧𝑒= 7

75

In Table 3.5, the average execution times reflect on the number of best solutions found by each metaheuristic algorithm. Reason being, each time the best solution had been improved upon, the counter for the idle number of iterations had been reset. As can be observed, the BPA and the eBPA best solutions were improved upon significantly more times than TS and SA. However, the BPA did find more improved solutions over that of the eBPA. The BPA and the eBPA were thus intelligent in finding more promising neighborhood regions within the confines of the solution space. This was followed by TS and then SA.

Table 3.6 gives the statistical values of the overall best and average fitness value solutions (i.e. BFV and AFV respectively). The 95% CI values are also given, along with the initial solution (IS). The fitness value refers to the total gross profit earned.

Table 3.6: Statistics of the best and average fitness values solutions, along with the 95% CI values

Methods BFV (ZAR) AFV (ZAR) 95% CI

IS 290,775,157 N/A N/A

BPA 338,353,400 338,349,798 AFV ± 725

eBPA 338,351,684 338,345,193 AFV ± 1,203

TS 338,340,881 337,493,100 AFV ± 261,742

SA 330,721,884 327,791,514 AFV ± 425,002

It is observed that each algorithm determined best solutions that improved upon the initial solution (IS). The BPA marginally determined the best BFV and AFV solutions over the eBPA, and had the lowest 95% CI value. This was then followed by the TS and SA algorithms. The BPA BFV solution determined a gross profit of ZAR 1,716, ZAR 10,803, ZAR 7,629,800 and ZAR 47,576,527 more than that of the eBPA, TS, SA and the IS respectively. Graphical comparisons of the metaheuristic statistics as given in Table 3.6 is seen in Figure 3.14 below. The 95% CI values are represented as the black interval estimates over the average fitness value towers.

76

Figure 3.14: The best and average fitness values, along with their 95% CI estimates

Visually, it is seen that the differences between the best fitness value performances of the BPA, eBPA and TS were minimal. Yet on average, the BPA and the eBPA performed significantly better than TS.

The BPA has also shown more consistency in having determined the lowest 95% CI estimate. This was only a marginal improvement over that of the eBPA. Having determined the best BFV and AFV solutions, along with the lowest 95% CI value, concludes that the BPA was the strongest and most consistent metaheuristic algorithm for this problem instance. However, the BPA overall performance was only marginally better than that of the eBPA for this continuous optimization problem.

The strengths of the BPA and the eBPA are attributed to their techniques employed in maintaining the solutions registered in their memory structures. The 𝑃𝐿 structures of both algorithms maintain a limited number of the best solutions found, at any given time, while traversing throughout the solution space. This maintenance is based on the idea of allowing solutions that meet the minimum criterion to be allowed admittance into the 𝑃𝐿 memory structures. The minimum criterion is that the fitness value of the worst solution must at least be improved upon with regards to the BPA, or at least be met with regards to the eBPA. If the admittance criterion of each algorithm were to be satisfied, then the design variables of the new solutions must be unique to be allowed admittance. Updates of the 𝑃𝐿’s are then performed by replacing the worst solution in the memory structures with that of the new.

Thereafter, for the BPA, the sorted order of the memory structure must be maintained. For the eBPA, the indices referencing the 𝑏𝑒𝑠𝑡, 𝑤𝑜𝑟𝑘𝑖𝑛𝑔 and 𝑤𝑜𝑟𝑠𝑡 solutions would need to be re-determined.

These techniques, along with the strategy of their probability factors in attempting to escape local entrapment, and the strategic reduction of the 𝑃𝐿 size for the eBPA, have shown to be an effective blend in traversing the solution space effectively for this problem instance.

0.326 0.328 0.33 0.332 0.334 0.336 0.338 0.34

BPA eBPA TS SA

Fitness Values (ZAR/Billion)

Best and Average Fitness Values with 95% CI

BFV AFV

77

Table 3.7: Statistical values of the irrigated water requirements (IWR) and the costs of production (CP)

Methods IWR (m³) CP (ZAR)

IS 244,491,000 156,924,202

BPA 241,997,367 154,799,322

eBPA 241,997,311 154,799,423

TS 241,998,185 154,799,348

SA 242,760,335 154,985,403

Table 3.7 gives the statistical values of the irrigated water requirements (IWR), and that of the costs of production (CP). As can be observed, each algorithm determined improved irrigated water allocation solutions over that of the IS. Interestingly, the CP values were also lower although the gross profit margins were higher.

From all algorithms, the eBPA determined a solution that required the least volume of irrigated water.

The eBPA determined a solution that required a volume of 2,493,689 m³ less than that of the IS. This was followed by the BPA, which required a volume of 2,493,633 m³ less. Thereafter, TS required a volume of 2,492,815 m³ less. Finally, SA required a volume of 1,730,665 m³ less. These solutions conform to the objective of yielding higher returns per unit of irrigated water consumed. At the quota of 9,140 m³ha^-1annum^-1, these savings would be able to supply irrigated water to an additional 272.83, 272.82, 272.7 and 189.3 hectares of agricultural land by the eBPA, BPA, TS and SA algorithms respectively. A visual representation of the irrigated water allocation solutions is seen in Figure 3.15 below.

Figure 3.15: Irrigated water requirements (IWR) of the initial solution (IS) and that of the metaheuristic solutions

0.2415 0.242 0.2425 0.243 0.2435 0.244 0.2445

IS BPA eBPA TS SA

Volume (m3/Billion)

Irrigated Water Requirements

78

Figure 3.16 shows graphical comparisons of the hectare allocation solutions. The BPA, eBPA and the TS show to have determined similar solutions. The metaheuristic solutions are also seen to be comparable to that of the IS due to the constraints of the lower and upper bound settings.

Figure 3.16: Comparison of the hectare allocation solutions per crop

The statistics of the hectare allocations (ha crop^-1), IWR, and the CP values of the initial and that of the best metaheuristic solutions are seen in Tables 3.8 and 3.9 below.

Table 3.8: Statistics of the initial (IS) and metaheuristic solutions per crop

Crops Methods ha’s crop^-1 IWR (m³) CP (ZAR)

Pecan Nuts

IS 100 1,155,300 597,153.143

BPA 50 577,650 254,826.6

eBPA 50.003 577,685.304 254,847.493

TS 50.001 577,662.84 254,834.181

SA 174.722 2,018,562.036 1,108,738.936

Wine Grapes

IS 300 1,497,600 1,849,889.52

BPA 499.971 2,495,856.1 3,210,253.1

eBPA 499.995 2,495,977.51 3,210,418.552

TS 499.751 2,494,757.158 3,208,755.529

SA 430.796 2,150,534.609 2,739,669.671

Olives

IS 400 3,021,200 2,114,959.24

BPA 750.029 5,664,967.7 4,096,961.8

eBPA 749.99 5,664,672.134 4,096,740.2

TS 750.215 5,666,375.011 4,098,016.827

SA 604.264 4,564,003.826 3,271,581.702

0 2000 4000 6000 8000 10000 12000

Hectares

Seasonal Land Allocations Per Crop Type IS BPA eBPA TS SA

79

Table 3.9: Statistics of the initial (IS) and metaheuristic solutions per crop

Crops Methods ha’s crop^-1 IWR (m³) CP (ZAR)

Lucerne

IS 7,500 75,022,500 40,722,449.25

BPA 7,000 70,021,000 37,122,431

eBPA 7,000.012 70,021,117.62 37,122,515.7

TS 7,000.033 70,021,327.18 37,122,666.53

SA 7,090.218 70,923,452.63 37,772,005.34

Cotton

IS 2,000 6,272,000 9,475,054.4

BPA 3,000 9,407,999.8 15,000,081

eBPA 2,999.988 9,407,960.899 15,000,012.71

TS 2,999.828 9,407,459.508 14,999,129.36

SA 2,987.453 9,368,653.092 14,930,759.94

Maize

IS 6,500 45,500,000 23,809,100

BPA 7,999.995 55,999,965 30,675,552

eBPA 7,999.944 55,999,604.87 30,675,316.6

TS 7,999.779 55,998,450.03 30,674,561.4

SA 7,986.315 55,904,203.44 30,612,928.84

Ground Nuts

CP 7,000 40,075,000 32,193,977.5

BPA 4,500.005 25,762,529 18,248,800

eBPA 4,500.069 25,762,894.54 18,249,155.67

TS 4,500.394 25,764,754.36 18,250,967.76

SA 4,526.232 25,912,678.59 18,395,095.89

Barley

IS 200 943,400 791,047.98

BPA 100 471,700.98 333,026.84

eBPA 100.001 471,707.002 333,032.689

TS 100.001 471,703.748 333,029.529

SA 224.294 1,057,994.541 902,319.617

Wheat

IS 12,000 71,004,000 45,370,570.8

BPA 12,100 71,595,699 45,857,390

eBPA 12,099.999 71,595,691.22 45,857,383.66

TS 12,099.999 71,595,695.3 45,857,387.02

SA 11,975.706 70,860,252.72 45,252,302.99

80