The parameter setting for the experiments, as Jamali (2009) considered, is given in Table 4.1 and 4.2. For the comparison of ILS approach with the existing literatures, we will refer to [Jamali (2009)]. In that dissertation the approach in [Morris and Mitchell (1995)]
denoted as SA_M (simulated Annealing (modified), and the approaches proposed in [Husslage. et al. (2006)] denoted as PD (periodic Design), SA (simulated Annealing) and MS denoted multi-start random generated approaches. We have also denoted the updated website values as Web (or Best known) values. The improvements obtained through the PD and SA approaches are discussed in Husslage et al. (2006).
Table 4. 1: Parameter setting for the experiments of ILS approach Experimental design LHD Perturbation Technique SCOE
Method ILS Stopping Rule MaxNonlmp parameter value
Optimal Criteria Opt(q5) MaxNonlmp setting 100 Local Move RP [Parameter,
, p 20
Acceptance Rule BI
-7
a
Table 4.2: The setting of number of runs (R) for the ILS approach
k N R
3-10 2-25 500
3-10 26-50 100
3,4,5 51-100 50
6-10 51-100 10
The experimental results of ILS approach regarding Euclidean measure [Jarnali, 2009] are given in the Table 4.3 and Table 4.4. We observe that ILS is able to detect a very large amount of improved solutions with respect to the best known ones. It is worthwhile to remark that for large (k, N) values the improvement of each LHD obtained by ILS approach is very significance. For the better visualization of the above results, Table 4.5 displays the summary of the performance of the several approaches. In the first row of Table 4.5 identical means, ILS approach able to identical solution compare to the best known results available in the literature whereas Worse means the solution obtained by ILS approach are worse compare to best known results. The performance of ILS approach regarding maximin LHDs in L2 measure is remarkable compare to other approaches available in the literature. This is, especially, true at large k values. For k? 6, with the exception of few numbers of low N values, all the solutions returned by ILS are better compare to the best known results. Though the performance of ILS approach is significantly better compare to other approaches consider here, but the approach will be effective if it is efficient i.e. the algorithm performs the job within acceptable time. So it is needed to comment about the computation times. It is worthwhile to mention here that there is no information regarding times to obtain the Web's results. Anyway for this demand, the computational cost of the approaches is reported in the Table 4.6. It is, however, quite clear that ILS is more computationally demanding with respect to PD and SA. Such higher costs are clearly rewarded in terms of quality of the results but the quality of the results might be wondered if the time restrictions are imposed on ILS. According to some further experiments that were performed, it would be realized that, especially at large k values, equivalent or better results with respect to the PD and SA ones, could quickly be reached by ILS. Therefore, it seems that at large k values even few and short nins of ILS are able to deliver results better than those reached by PD and SA. That is ILS approach outperforms compare to other approaches.
Table 4.3: Comparison among PD, SA, Web and ILS approaches regarding maximin Li-IDs in Euclidean distance measure for k3 - 6
u.pl
II tj
•i II II 1,1 I.1II •
'
_Ii _II 1 I(I
p4
ti 'II 'kJ II iiI
ELW1 _•u-ii ii _ic
•lJI _:.__
tI fI
I!3 .ri _
_
um
11J iwiir
on on EMM
on
moms
ism
56
Table 4.4: Comparison among PD, SA, Web and ILS approaches regarding rnaximin LHDs in Euclidean distance measure for k = 7- 10
A.7 6-9 6-9 8'.))!
N
1.13 SA v,, 1.9 SA ILS 47, ILS SA \9'), ILS
7 7 -i 7 0 3 3 9 9 9 0 I))
7 0 I) 5 13 14 4 6 18 8 9 19 19
16 2) 2> 2) 26 20 26 2% 28 25 35 13 33
E
In 32 52 37 40 10 40 43 43 43 50 50 50
29 47 47 47 54 54 54 6) 61 61 68 68 60
3! 1.! 6I 6I 70 70 77 0)) 81) 8I 89 09 90
46 79 79 71) '5) 90 91 07 0) *02 114 114 11$
-- - 47 92 92 93 112 112 II) 126 126 129 141 142 1.43
*0 68 110 II)) III 71) II! 33 54 54 57 72 172 74
II 1,9 1 19 59 32 57 752 154 76 I 16 1 1;) 766 706 209
42 95 IS)) 132 159 176 177 IA! 21)4 21)4 21)9 235 235 240
II 95 174 170 IS! 2111 206 210 212 235 242 261 1 268 275
14 119 204 219 217 220 230 243 265 260 279 296 1 905 313
15 729 2)! 221) 223 251 77) 2%)) 296 10') 37% 337 347 368
II, 55 238 24) 249 286 317 326 1311 352 .35% .670 1 393 404,
7 II 256 256 272 512 182 412 867 556 405 415 1 442 458
18 IX6 26! 291 298 344 361 .363 398 451 466 45% 496 509
95 505 323 326 37)) 390 398 438 469 472 490 554 509
226 392 349 403 425 434 472 306 517 542 675 (.4!
236 .181 380 393 436 4),) .87) 317 549 959 592 650 050
27)) .47)4 41 8 425 467 5))) 900 355 590 4 1 4 60) I 691 7)14
R24
273 4)11 446 454 50 1 542 5-19 096 640 (.5! 605 7.17 75))
1110 444 491 492 33% 5145 555 658 400 6)19 770 000 6110
35)) 467 020 53! 583 676 637 690 739 752 797 657 875
163 419 548 612 664 6674 726 79) 41!)! 834 I 'III) 95!
382 526 5S5 59') 64% 1)2 733 756 840 859 896 976 101)2
_ 406 54) 620 634 693 7471 785 326 800 919 95.34 1041 1061
417 053 654 675 737 917 827 876 956 936 11115 11 06 132
30 450 62)) 691 714 787 049 897 925 1)3)9 10-I) 1090 1 1173 1207
.7 482 697 728 764 812 9)) 976 1104 11 114 1116 174) 279
32 5 1 8 699 170 803 866 966 976 1024 1179 176 1194 1 131$ 1351
33 357 727 814 944 91)0 1010 1017 0104 1291 1244 1253 1 1416
561 751 851 88) 945 1072 11)0" 1130 1270 316 329 1 1.173 151.4
35 566 811 0(4 934 7002 II)) 1 151 III)) 1.326 139X I 550 15.95 4095
36 636 all 91' 969 1042 1TW1 1205 • T7 1405 1444 1459 1647 *679
37 ('08 863 97). *012 1079 1230 1272 1300 1477 1 305 1516 (72) 4761
38 709 923 1028 *050 1123 12716 1328 1867 1534 1 577 1597 ('9)) *832
39 726 939 084 *094 092 1344 397 434 101)9 640 (1.05 (070 1937
40 786 970 1122 414$ 1223 l.iIN 451) 14w, 1675 1728 1742 1 1"46 211)1
41 007 1016 1156 1197 1271 1496 1 535 1562 1763 1793 (970 1 765% 2135
42 901 1 1)64 7209 4249 11.11 1926 1584 611) 1041 1071 1920 1 2149 2491
43 903 1112 1256 13)1) 1377 1397 1635 1683 1905 1957 1973 222.1 2279
44 003 114)) 1136 1340 14,13 1693 1698 1752 1994 2042 7072 230 - 2573
45 926 1102 1366 4400 1480 1721 *155 1821) 2079 2126 2)0) 74(5 2466
48 953 1243 1408 448 1540 794 1819 1981. 2153 222)) 2209 25117 2368
47 99% 1 266 1451) *521 1016 1947 18$1 1939 2244 2.0)2 21)) 7)-00 2663
40 1054 1520 15)) *578 165% 1924 1957 2017 2336 2383 2307 2732 2760
49 (1)74 (156 I1582 1645 1729 7009 21318 2101 20*7 247)) 747)1 2875 2980
50 1113 1 1911 1639 1699 1772 20.11 2009 2)79 2497 7569 2556 215) 2991
51 1161 7-150 1602 *744 7855 2112 2152 7241 2566 2637 2611 5006 11)91)
52 725) (487.] 734 1804 _18 220) 2218 2325 2680 2716 7/45 Sill 3207
53 1241 7537] 1081, 1949 2214 2288 2429 2713 2798 2829 02)1) 3306
54 208 517
18 856
8 0
11*32 2006 2356 2083 2473 2008 2894 2092 9399 3412
53 7325 (1.5(1 894, 2000 2003 1 2371) 1 2462 257)) 293' 2951, 51154 3492 353)) _ (.150 1701 1 2))1)) 2073 2162 12444 1 2511 2624 .11121 5040 91)111 I 1551 4643 57 7479 1(2) 207-4 2)193 219.) 1 259.1 1 2670 2703 3)19 3162 5215 37.51 3763
SR 479 17,lq 1 2043 2156 3239 2(10) 2671) 2706 3)87 5265 9803 87)5 .4841
59 7509 1011 2)36 2167 7356 27)5 2793 330) 3297 3339 3399 1590 3977
61) 577 IS'!') I 2252 2277 23')) 2796 2573 71)15 342)) 5346 5900 $115)) 4)6')
1'I2S 2701, 2316 2489 1666 2966 807) I 8628 5565 5906 4)95 3202
62 690 2)121 ' 2187 2931 2977 31340 3152 3656 31,9I 3700 -I)))
63 168)1 2)154
73.1 6
237 2477 _ 07 3055 3)6)4 3215 569)) 07141 3761 1 4 _59 44')1
21)7)3 2452 2483 2734 801)7 3207 3292 1 3820 3869 3955 45).) -49(10)
65 1796 2152 7302 2547 2721 320 3296 15 5 7 3112 19'!) 41134 4581 4698
66 857 2150 7545 21.06 204) 3259 9-ItS 1474 4111)4 4)184 0141 4769 4610
67 186$ 2215 7038 2972 2068 3)39 3488 5543 40%) 4200 1 4223 49-17 4)0)
68 194)) 229) 2603 2714 2956 345) 3600 1637 4212 4517 4360 41l')c 5077
2_ 1953 2191 2746 2794 71)73 552)) 3704 3716 4517 440)) 4355 9)27 -522)
71) 21.1)) 2417 2380 2951. 1)40 0568 3775 384) 4364 .45)6 4930 3278 5366
7! 7150 7131 7971 2919 3!6) 3740 3877 3934. 4548 4666 4605 5417 5479
72 21 77F 290) 2014) 2132 327)) 3)111) 3562 4027 4666 4758 48(2 9366 -5675
7) 7706 2)90 30-12 3077 131)5 8037 400') 113.4 4776 4099 087) 565) 9746
74 22.).) 76)4 5)2)) 3117 14)2 ' I $127 427.4 49) S 1' '1017 3030 5017 507')
721.9 77(131 3157 3230 3313 4073 32)3 4294 51)1)5 5141 1 0171 5557 60)5
76 7315 2798 1 3210 3289 5539 4)78 4427. .1)9) 917° 71241 1 2'.) 1 7,))I 0191
77 25)5 1 3520 859') 51,17 4764 4384 4447 5722 .35(4 5910 4272 6305
79 29))) 2570 I 3412 5064 4991) 449! .1)77 1)56 95-13 545'l 0354 64-49
2_ 7 5 25 20,,, *414 4408 3775 4413 4985 47)15 5556 1 56.)) 1 31,33 0)7,5 i!'
0)) 2590 2.17') 1 356.1 1 3877 157,9 1 4495 1 1807 3577 1 3707 .1,55 (.7)3
81 7642 1 81757' 1 361') 34176 411)11 4679 '*721 4550 3741 3922 5)01 (.7711) -68-42
2753 3 IS 947.9 3127 3598 42)9 43))') -7 5659 (.18-1I ((1118 60)5 70.11
83 7767 0)15 5771 30110 4076 4045 4906 51112 5074 61 96 40)7 7)l'14 -7250
2855 912 5 907)) 3055 4)3) 9006 5722 611') 6357 7757, 7362
83 20 14 11111 1919 3994 4324 51)02 9III) 033)) 62 I 7 647') 61')) [ 7157 -19)8
86 11)3 51)) 3955 -*052 4)97 5164 5705 5425 6340 (.61)6 1 6491 [ 7592 -74.717
87 3 103, 3450 40(8 4)11) 4.134 572' 33)12 53)8 6)91 6761 5972 1 9'I 7637
89 93 15)111 3(06 .4)9') .4524 5540 5421, 5667 (.66)) (.071 I 00) 1477 -9022
89 31XI 95-1) 3)7). '*2)1)) .4576 5.198 5519 1 6774 6790 71)0.) 1 6077 1 71l11) XIII
90 91)
- 1881 31118 -* 3 02 40)3 7370 5608 3092 6901 7172 7(141) [ 5770 -5325
0) .2)) 3677 4371) -4.421 4050 5676 5696 5089 6931) /29(, 7)63 1 33.111 0464
92 3271 970)) 3370 4521. 4319 5795 5822 1 6091 1 704,7 1.596 7256 0447 - 01)51
_2_ 3)6) 40)) I117 4374 4504 5837 5925 6231 7947 74.16 7-150 1 061)) 00276
94 3474 35.55 47.9) *679 5087 61(1)7 1,032 1,47') 1 7437, 74.47 r 5710 9)167,
2] 95.)) II.). 4703 5)54 (6(.4 1)44 7.506 748'! 7740 77-11 0517 - 92 9 1
06 3634 0 .1060 L 4562 5220 ((222 6227 (.5)6 76-15 7926 7777 51-14 94-IS
______ 4040 '14111 53)6 v.104 1,764 '"t'4-T ,78I 071!) 00(3 '1554 - 000))
31.50 17[ 4036 [ 5)107 5440 6376 (-167 1,771. 7496 5)52 0747 915)
_22_. 3781 -121-I [ 4519 1 ' ' 577 (.97) - 1.'112 J 8621 0250 - 7.540 1 06!7 L. 1011],J, 4)() , 5048
-- S 104 1 "Li _..22~. L .i1'.L_ - I! L 'iii...._.
I.
Table 4.5: Summary of the comparison among several approaches of finding maximin LHDs for N=2 to 100
Number of best solutions (niaxirnin_LHD) Identical Worse
k PD SA SA_ M Web MS ILS ILS ILS
3 ]61} 0
J
0 65 0 14[
20 654 ]02} 0 0 47 0 34 18 47
0 0 II 0 'is 10 11
6 OOj 0 0 00 0 90 09 00
7 00 0 0
f
00 0 92 07 008 0
[
0}
00 0 93 06 009 0
[
0 00 0f
93 06 0010 0
[
0 00 0}
92 07 00Table 4.6: Comparison of computational cost TotalElansedTime thrs
k PD SA ILS
3 145 500 IM
4 61 181 507
5 267 152 767
6 108 520 1235
7 232 246 698
8 - 460 846
9 -- 470 1087
10 -- 470 1166
Figure 4.1: Muliticollinearity analysis of the LHDs obtained by ILS approach
58
From the above discussion it is clear that ILS approach is state-of arts for optiinality analysis regarding Euclidian distance measure as well as computational cost. Apama (2012) also analyzed the performance of ILS approach regarding multicollinearity of the optimal LHD measured in Euclidean distance. The experimental results regarding average correlation are given in the Figure 4.1. It is noted that the average coefficient of correlation are calculated as define in [Aparna 2012]. We observe that, except few LHDs, the average coefficients of correlations among factors are less than 0.2. It may conclude that the optimal LHDs optioned by ILS approach regarding Euclidean measure have poor multicollinearity i.e. among the factors of each LHD exists good orthogonality property.
It is also remarkable that the avarage coeffient of correlations are decreses with the increases of number of factors. It is whorthwhile to mention here that the performance of ILS approach is increase with the increase of factors as well as incerasing of number of design points (see Table 4.5).
CHAPTER 5
Optimality Analysis and Discussion of the Experimental Results Regarding Manhattan Distance