In this section, we evaluate the performance of the proposed ILP formulation presented in the earlier section.
Experimental Setup: The experiments have been conducted using benchmark PTGs adopted from [1, 2, 60]. In particular, we considered two real-world applications namely, Gaussian Elimination and Epigenomics. The PTG representation of Gaussian Elimination and Epigenomics are shown in Figures 4.5a and 4.5b, respectively. For the scheduling of these two PTGs on a heterogeneous distributed platform, we have varied the following parameters: (1) Number of processors p = {2,4,6,8}, (2) Com-
(a) (b)
Figure 4.5: (a) Gaussian Elimination [1], (b) Epigenomics [2]
munication to Computation Ratio CCR = {0.25,0.5,0.75,1,2} (CCR is the ratio of the average communication cost to the average computation cost. That is, CCR =
1
|E|Σmij.(n×p1 [Σni=1Σpr=1ei1r])). (3) Number of service-levels of each task Ti is taken as,
|SLi|= 3, (4)Execution time ei1rof each task nodeTiat its base service-level for proces- sorPr, is taken randomly from a uniform distribution within the range 10 msto 30ms.
The execution time (eilr) for non-base service-levels (starting from level 2) of the tasks are assigned uniform random values bounded between 110% and 130% of the execution times (ei(l−1)r) corresponding to their immediately lower service-levels, (5) The rewards (QoSil) for any taskTi, increase monotonically as service-levels become higher. The val- ues of the rewards have been chosen randomly from the range 1 to 200, while ensuring that the random reward value for a task at a given service-level is higher than the reward values at lower service-levels. (6) Communication time mij corresponding to each edge in the PTG has been generated from a uniform random distribution within the range 10 ms to 30 ms. The obtained communication times are then appropriately scaled to maintain desired CCR, (7)Deadline for a PTG is obtained from the makespan outputs computed by applying the list scheduling based heuristic scheme PEFT [1] on the given PTG. In particular, we compute two makespan DL and DH by setting all task nodes at their base and highest service-levels, respectively. Finally, the actual deadlineD for the
PTG is randomly selected from a uniform distribution in the range [DL, DH]. All exper- iments are carried-out using the CPLEX optimizer [10] version 12.6.2.0, executing on a system having Intel(R) Xeon(R) CPU running Linux Kernel 3.10.0-693.21.1.el7.x86 64.
Performance Metrics: Four metrics have been used for evaluating the designed ILP based scheduling strategies: (1) Normalized Reward: N R (in %) = RRACT
M AX ×100,
where,RACT is the actually obtained reward andRM AX is the maximum possible reward for the PTG. (2) Deadline extension Rate (DR) determines the extended deadline of a given PTG as: D = DL+ ((DH −DL)×DR), where DR ∈ {0,0.25,0.5,0.75,1.0}. For example, the different extended deadlines corresponding to various values of DR for a PTG with DL = 20 andDH = 40 are 20, 25, 30, 35, 40. (3) Running time (in seconds):
Total time taken to compute the solution for a given PTG. (4) Percentage of tasks upgraded: Given a PTG and an inputx (∈ {0,25,50,75,100}), we have selected x% of task nodes in the PTG and upgraded them to their highest service-levels. Specifically, x% of the tasks which have the highest reward per unit execution time (RPE) values have been chosen for service-level enhancement. Here,RPE of a taskTi is defined as the ratio of the difference in reward to the difference in execution time, whenTi is upgraded to the highest service-level from its base level.
70 80 90 100
0 0.25 0.5 0.75 1
Normalized Reward (in %)
Deadline Extention Rate
#Processor = 2
#Processor = 4
#Processor = 6
#Processor = 8
(a)Gaussian Elimination
70 80 90 100
0 0.25 0.5 0.75 1
Normalized Reward (in %)
Deadline Extention Rate
#Processor = 2
#Processor = 4
#Processor = 6
#Processor = 8
(b) Epigenomics Figure 4.6: Effect of varying processors.
Experiment-1: Varying the number of processors: In this experiment, we vary
the number of processors (p) from 2 to 8, while fixingCCRto 0.5. Figure 4.6 depicts the results for this experiment. It may be noted that for any given deadline, the normalized rewardN R increases as the number of processors becomes higher. This happens because the residual system capacity increases with increasing #processors and this capacity is utilized by the system in order to enhance task service-levels, resulting in higher N R values. For example, in Gaussian Elimination PTG with DR = 0.25 (Figure 4.6a),N R values for p = 2 and p = 6 are∼87% and ∼90%, respectively.
60 70 80 90 100
0 0.25 0.5 0.75 1
Normalized Reward (in %)
Deadline Extention Rate CCR = 0.25
CCR = 0.5 CCR = 0.75 CCR = 1 CCR = 2
(a)Gaussian Elimination
60 70 80 90 100
0 0.25 0.5 0.75 1
Normalized Reward (in %)
Deadline Extention Rate CCR = 0.25
CCR = 0.5 CCR = 0.75 CCR = 1 CCR = 2
(b) Epigenomics Figure 4.7: Effect of varying CCR
Experiment-2: Varying CCR: We vary CCR from 0.25 to 2, while fixing p to 2. Figure 4.7 depicts the results for this experiment. For fixed values of p, n and D, higher values of CCRimply lower computation demands of the task nodes on processor resources at any service-level. Such lower computation demands in turn, naturally en- hances the possibility of task service-level upgradation. Consequently, this leads to an increase in the obtained rewards, N R. For example, in Epigenomics PTG with DR = 0.25 (Figure 4.7b), the normalized rewards obtained forCCR = 0.25 andCCR = 2 are
∼86% and ∼96%, respectively.
Experiment-3: Comparing ILP-SATC and ILP-SANC: We set p to 2 and CCR to 0.5. The total amount of time taken by both ILP-SATC and ILP-SANC, when the deadline is varied from DL to DH, is shown in the Table 4.5. Observing the
PTG ILP Version Deadline Extension Rate
0 0.25 0.5 0.75 1
Gaussian Elimination
ILP-SATC 16m:30s 2h:46m:43s @ @ @ ILP-SANC 2.6s 2.7s 1.6s 0.6s 0.4s
Epigenomics ILP-SATC @ @ @ @ @
ILP-SANC 45.7s 55s 23.1s 2.3s 0.4s
Table 4.5: Running time of ILP-SATC and ILP-SANC. The symbol @ represents running times greater than 24 hours
results obtained for the Gaussian Elimination application, it may be clearly seen that run-times for ILP-SATC significantly increases with larger deadlines. This indicates ILP-SATC’s strong dependence on the value of the deadline considered. On the other hand ILP-SANC exhibits drastically lower run-times for all cases.
30 40 50 60 70 80 90 100
0 0.25 0.5 0.75 1
Normalized Reward (in %)
Percentage of Tasks Upgraded ILP-SANC
PEFT
(a)Gaussian Elimination
40 50 60 70 80 90 100
0 0.25 0.5 0.75 1
Normalized Reward (in %)
Percentage of Tasks Upgraded ILP-SANC
PEFT
(b) Epigenomics Figure 4.8: Comparison with PEFT [1]
Experiment-4: Comparison with the state-of-the-art: This experiment com- pares ILP-SANC against the list based heuristic scheduling algorithm PEFT [1]. The essential objective of PEFT is to minimize makespan corresponding to a task graph in which all tasks have only a single service-level. Therefore, as PEFT is task service-level oblivious, in order to apply PEFT within our framework, service-levels of all tasks must be fixed before its application. After assigning selected service-levels to task nodes,
PEFT is run on the PTG and the resulting normalized reward N R and makespan val- ues are noted. ILP-SANC is then executed on the same PTG with the makespan value delivered by PEFT, as deadline. To improve normalized reward values for PEFT, we have selectively chosen higher service-levels for those tasks which deliver higher reward gains with respect to additional execution time consumed; that is, tasks with larger RP E values have greater priorities towards higher service-level assignment. The exper- imental results are depicted in Figure 4.8. It may be observed that our proposed ILP based scheme is able to achieve higher normalized rewards compared to PEFT for any given deadline bound, unless the deadline is so relaxed that PEFT is also able to assign highest service-levels to all tasks.