The Proposal of Demand Estimation of Repairable Items for the Weapon Systems
3.8 Application
3.8.1 Approach-I: (Classic Approach)
Approach-I takes the all the data set and tries to determine the best scenario for all.
In order to test whether there is a difference among 24 scenarios within each forecast period in projecting the number of failures, the Friedman test is conducted. Based on the test result shown in Table 3.6 below, it is concluded that there is a significant
11 The Friedman test is the nonparametric equivalent of a one-sample repeated measures design or a two-way analysis of variance with one observation per cell. Friedman tests the null hypothesis that k related variables come from the same population. For each case, the k variables are ranked from 1 to k. The test statistic is based on these ranks (IBM-SPSS Statistics Base 17.0 User Guide)
12 The Wilcoxon signed-rank test is a nonparametric statistical hypothesis test used when compar- ing two related samples, matched samples, or repeated measurements on a single sample to assess whether their population mean ranks differ (i.e. it is a paired difference test). It can be used as an alternative to the paired Student's t-test, t-test for matched pairs, or the t-test for dependent samples when the population cannot be assumed to be normally distributed. (http://en.wikipedia.org/wiki/
Wilcoxon_signed-rank_test) Table 3.6 Friedman test results
Forecast period N Χ2 Degrees of
freedom p-value
1-year forecast
(2007/4–2008/3) 894 35.1 23 0.050*
2-year forecast
(2007/4–2009/3) 894 164.8 23 0.000*
3-year forecast
(2007/4–2010/3) 894 266.9 23 0.000*
* p-value=0.05
difference among the scenarios within each forecast period in projecting the num- ber of failures at the α = 5 % such that for 1-year forecast period; (Friedman Test:
χ2 (23) = 35.1, p = 0.050), for 2-year forecast period; (Friedman Test: χ2 (23) = 164.8 and p = 0.000), and for 3-year forecast period; (Friedman Test: χ2 (23) = 266.9 and p = 0.000).
The mean and variances of scenarios along with the results of Friedman Test are presented in Appendix-I. As seen in Appendix-I, Scenario RDS (1) came out to be the best scenario in considering the forecast period of 1, 2 or 3-year. Scenario RDS (1) is compared with the Scenario RDS (3) and TOIMDR (8) in terms of mean and presented in Table 3.7 below.
To determine which scenarios are statistically different in projecting the number of failures at the α = 5 %, Wilcoxon Sign Test is run and significantly different ones are listed in Appendix-II. The first comparison in Appendix-II is given as example.
Scenario SORTIE (7) is significantly different from Scenario TOIMDR (7) in pro- jecting the number of failures at the α = 5 %, (z-value:−3.59, positive rank, and p- value: 0.000). The other results can be interpreted likewise.
3.8.2 Approach-II: (New Approach-I/NSG Based Approach)
Approach-II takes the data based on NSG and determines the best scenario for each NSG. The research question in this approach is the following: “Is Scenario RDS (1) the best in projecting the number of failures for all NSGs at the α = 5 %?” So, the data set of 894 is divided into the 16 NSGs and Friedman test for each NSG are run in order to test whether there is a difference among 24 scenarios within each forecast period in projecting the number of failures. The best four or five scenarios at most are determined based on mean, standard deviation, χ2 and p-value at the α = 5 %. The scenarios that are recommended for projecting the failures are listed in Appendix-III. In answering the research question for 1-year forecast period, Sce- nario RDS (8) for NSG-10 and scenario SORTIE (1) for NSG-15 are the only ones that are significantly different from the RDS (1) in projecting the failures at the α = 5 % (Freidman Test: χ2 (23) = 49.80, p = 0.001 and χ2 (23) = 70.54, p = 0.000, re- spectively). Scenario RDS (1) seems to be the best for the other NSGs for 1-year forecast period. The other results can be interpreted likewise.
Table 3.7 Comparison of algorithms’ results-1
Forecast period RDS (3) TOIMDR (8) RDS (1)
1-year forecast
(2007/4–2008/3) 4.21 4.6 3.9
2-year forecast
(2007/4–2009/3) 8.06 8.98 7.45
3-year forecast
(2007/4–2010/3) 12.05 12.95 11.61
The Wilcoxon Sign Test is conducted for those scenarios that are determined as a significantly different during the Friedman Test and the results of Wilcoxon Sign Test are listed in Appendix-IV. Based on test results, the one that is significantly dif- ferent from the other one is marked with “**” in Appendix-IV. The NSG-10 data is given as an example for the 1-year forecast period in Appendix-IV. Recall that five scenarios are recommended in Appendix-III such as RDS (8), RDS (7), SORTIE (8), RDS (5) and RDS (6). As shown in Appendix-IV, the pairwise comparisons between RDS (1) and RDS (8) [Wilcoxon Test: Z-value:-2.386, positive rank and p-value: 0.017], RDS (1) and RDS (6) [Wilcoxon Test: Z-value:-2.232, negative rank and p-value: 0.026] and RDS (6) and RDS (8) [Wilcoxon Test: Z-value:-2.043, negative rank and p-value: 0.041] are significantly different in projecting the num- ber of failures at the α = 5 % for 1-year forecast period. The other pairwise compari- sons in NSG-10 are not resulted in significantly different. The other results can be interpreted likewise.
3.8.3 Approach-III: (Extension of Approach-II, NSG Based Approach)
Approach III uses the result of previous approach by taking the best scenarios for each NSG and forecasts the projected failures. This approach requires the using the RDS (8) for NSG-10, SORTIE (1) for NSG-15 and RDS (5) for NSG-49 and RDS (1) for the rest as indicated in Appendix-III at the 1-year forecast period. It can be interpreted likewise for 2 and 3-year forecast period. As seen in Table 3.8 below, the progress is almost 1 % when compared to the scenario RDS (1) based on average absolute error.
3.8.4 Approach-IV: (New Approach-II/Item Based Approach)
Approach-IV takes the each item in the data and determines the best scenario for each item. In order to display the difference of this approach from the previous ones, the first 5 NSN of NSG-10 are taken as an example and shown in the Table 3.9 below.
Table 3.8 Comparison of algorithms’ results-2
Forecast period RDS (3) TOIMDR (8) RDS (1) New Aproach-I
1-year forecast
(2007/4–2008/3) 4.21 4.60 3.90 3.85
2-year forecast
(2007/4–2009/3) 8.06 8.98 7.45 7.39
3-year forecast
(2007/4–2010/3) 12.05 12.95 11.61 11.47
As seen in the Table 3.9 above, RDS (1) gives the minimum average absolute er- ror for overall data set in classic approach. But when taking NSG specific approach, RDS (8) and RDS (6) appear to be the best scenarios that have minimum average absolute error. When taking item specific approach, the first two minimum results, IBA (1) and IBA (2), show that there are different scenarios appear to be the best in addition to RDS (8) and RDS (6).
The results of average absolute errors for IBA (1), IBA (2), IBA (3), IBA (4) and IBA (5) are given in Table 3.10 below. It is seen in the Table 3.10 that even the aver- age absolute error of IBA (5) yields better result that that of RDS (1).
The research question in this approach is the following: “Are Scenario IBA (1) and IBA (5) significantly different from the scenario RDS (1) in projecting the num- ber of failures at the α = 5 %?” In answering the research question Wilcoxon Sign Test is conducted for just 1-year forecast period as an example. The pairwise com- parisons between IBA (1) and RDS (1) [Wilcoxon Test: Z-value:-22.496, positive rank and p-value:0.000], IBA (5) and RDS (1) [Wilcoxon Test: Z-value:-10.312, positive rank and p-value: 0.000] are significantly different in projecting the num- ber of failures at the α = 5 % for 1-year forecast period. It is concluded based on the previous pairwise comparisons that IBA (2), IBA (3) and IBA (4) are significantly different from RDS (1) in projecting the number of failures at the α = 5 % for 1-year forecast period, too.
3.8.5 Evaluation
The results for scenarios are presented in Fig. 3.1 and Fig. 3.2 based on average ab- solute error and average percentage error, respectively. It is seen that the sequence of the two demand prediction parameters are the same but the value of the average percentage error is relatively larger than the other parameter. Using the best sce- narios that are close to the actual values let the decision maker make more effective buy decisions. In order to show the high acquisition and repair cost value of F-16 repairable items, Table 3.11 is prepared. It is clear to have the conclusion that im-
Table 3.9 Best Scenarios for NSG-10
NSN Classic NSG Based IBA (1) IBA (2)
NSN-1 RDS (1) RDS (8), RDS (6) TOIMDR (5) TOIMDR (4)
NSN-2 TOIMDR (8) SORTIE (8)
NSN-3 RDS (7) RDS (8)
NSN-4 RDS (7) RDS (6)
NSN-5 SORTIE (7) TOIMDR (8)
IBA item based approach The scenarios that have the smallest average absolute errors are ordered up to five such that IBA (1) indicates the smallest, IBA (2) indicates the second smallest, IBA (3) indicates the third smallest scenario with respect to average absolute term. IBA (4) and IBA (5) should be read likewise
proving the average absolute error by one or two unit per repairable item yields a great impact on budget based on the high mean value of repairable items.
The sum of the difference between the actual repair cost per item and the pro- jected repair cost based on the scenario result per item is shown in Fig. 3.3. It is clear that using scenario results found in this study saves almost 2–6 million TL.