5.3 Experiments and Results
5.3.1 Simulation Results
A series of experiments have been conducted in order to measure the performance of AVSA in terms of different aspects of the QoE achieved by the system under varying scenarios. The performance achieved by the AVSA framework using the three proposed adaptation strategies, namely, (i) the DP-based Quality-level Al- locator (DPQA), (ii) the Streamlined DP-based Quality-level Allocator (SDQA) and (iii) the Approximation Algorithm (SDQA-AA) at distinct values of approx- imation quality index (m), have been measured. Additionally, AVSA is com- pared with a popular adaptive streaming framework called AVIS [17] using its two adaptation strategies, namely, (i) Dynamic programming approach for AVIS (AVIS-DP) and Greedy approach for AVIS (AVIS-GREEDY).
Figure 5.5 depicts plots forBuffering %. It may be observed thatBuffering% is approximately same for all the quality level allocation schemes (DPQA,SDQA andSDQA-AAat distinct values ofm) of theAVSAframework. It is also approxi- mately constant with increasing total number of video flows (system load). Main- tenance of stable buffer sizes is achieved mainly by a combination of two principal mechanisms: (i) Throttle rate adjustment for each flow based on instantaneous buffer size by TRC and (ii) Cell capacity cum traffic load aware quality level
5.3 Experiments and Results
2 4 6 8 10 12 14 16 18
10 20 30 40 50
—B—u—f—f—e—r—i—n—g %
#Video Flows DPQA SDQA SDQA-AA (m=5.0) SDQA-AA (m=10.0) AVIS-DP AVIS-GREEDY
Figure 5.5: Buf f ering % Vs.
#Video Flows
32 34 36 38 40 42 44 46 48 50
10 20 30 40 50
—P—S—N—R
#Video Flows
DPQA SDQA SDQA-AA (m=5.0) SDQA-AA (m=10.0) AVIS-DP AVIS-GREEDY
Figure 5.6: P SN R Vs. #Video Flows
allocation by AVRC. On the other hand, both the adaptation strategies of AVIS (namely, AVIS-DP and AVIS-GREEDY) encounter higher Buffering%. This is because, their adaptation strategies are ignorant of client-side buffer status.
Fig 5.6 depicts the plots for the average transmitted video qualities (PNSR) by the different quality level adaptation strategies, as the number of video flows vary from 10 to 50. It may be noted that as a consequence of being adaptive, the trends for PNSR for all the adaptation methodologies are decreasing as the number of flows (system load) increases. This is expected because the average number of RBs that may be allotted for a video flow reduces as the total number of video flows increases under a fixed resource block budget within an adaptation interval. With a lower number of available RBs per flow, buffer outage events may only be controlled by degradingquality levels of the flows. Although the trends for all the adaptation algorithms in Figure 5.6 are similar, PNSR achieved byAVIS- GREEDY is lowest among all the adaptation strategies due to greedy decisions in its quality level/bit-rate selection process. On the other hand, the AVIS-DP, DPQA, SDQA strategies provide highest and approximately same PNSR due to optimal decisions in their adaptation process. However, a closer look reveals that AVIS-DP slightly outperformsDPQAand SDQA. This is expected because DPQA and SDQA operates within AVSA which also attempts to maintain a stable playout buffer sizes for all clients by appropriately adjusting their throttling
Table 5.3: Comparative results for average run time (in millisecs)
Video DPQA SDQA SDQA-AA AVIS-DP AVIS-GREEDY
Flow m = 2.5m= 5.0m = 10.0m = 15
10 1282.0 15.6 3.0 1.0 1.0 0.055 1511.0 0.012
20 3648.1 131.5 12.5 6.6 3.9 3.000 4227.7 0.018 30 6669.0 387.6 28.1 15.8 9.2 7.727 8498.1 0.026 40 10247.1 836.2 44.7 24.3 14.7 12.018 11697.6 0.030 50 14468.8 1474.6 66.3 36.4 22.1 18.618 16459.3 0.036 Considered system with one core and 2.5 GHz processing capacity
0 200 400 600 800 1000 1200 1400
10 15 20 25 30 35 40 45 50
Speed-up
#Video Flows
SDQA SDQA-AA with m = 02.5 SDQA-AA with m = 05.0 SDQA-AA with m = 10.0
Figure 5.7: Speed-up Vs. #Video Flows
0 1 2 3 4 5
10 15 20 25 30 35 40 45 50
% Loss in Avg Video Quality
#Video Flows SDQA SDQA-AA with m = 02.5 SDQA-AA with m = 05.0 SDQA-AA with m = 10.0
Figure 5.8: % loss in video Quality Vs. #Video Flows
rates. Additionally, it may be observed from Figure 5.6 that the SDQA-AA adaptation scheme provides slightly reducedPNSRwith respect to theDPQAand SDQA algorithms due to approximation in the memoization process. Moreover, the performance of SDQA-AA degrades with increasing values of approximation quality index (m). Such degradation occurs because, with larger values of m, SDQA-AAapplies a higher degree of approximation in the memoization process.
Table 5.3 shows the average run times (measured in millisecs) taken by the DPQA,SDQA,SDQA-AA,AVIS-DP andAVIS-GREEDY strategies as the num- ber of video flows vary from 10 to 50. It may be observed from the table that the average execution time of DPQA and AVIS-DP are comparatively much higher than the other strategies. This is because both these strategies are based on conventional dynamic programming which calculates and memoizes partial so-
5.3 Experiments and Results
lutions for all possible bounds on number of flows (∀i ∈ [0, N]), quality levels (∀l ∈ [xi, yi]) and RBs (∀α ∈ [0, R]). On the other hand, the SDQA scheme memoizes only those partial optimal solutions which provide distinct enhance- ments in quality with increment in the bound on the number of RBs α, for each value of i. As a result, SDQA provides significant reduction in computational overhead. In comparison, SDQA-AA achieves further significant reductions in execution time due to the approximation applied. As the value of the approxima- tion quality index (m) increases, theSDQA-AA strategy is able to obtain higher reductions in the number of memoized partial solutions, which reflects as lower required run time. The average run time taken by the AVIS-GREEDY strategy is lowest among all the adaptation strategies. This is because, the computational complexity of AVIS-GREEDY only depends on the available number of flows and the number of available quality levels. Fig 5.7 portrays the execution speed- ups achieved by SDQAand SDQA-AA (at distinct values of m) over theDPQA algorithm, as the number of video flows vary from 10 to 50.
On the other hand, Figure 5.8 depicts the plots for percentage loss in average video quality suffered by SDQA-AA (at distinct values of m) with respect to DPQA, as the number of video flows vary from 10 to 50. It may be observed that although as expected, the average performance of SDQA-AA degrades with increasing values of m, the degradation is not drastic. For m = 5, the loss in average video quality for SDQA-AA is less than 1% even for 50 users while the speed-ups obtained are more than ∼400 times.
Figures 5.9(a), 5.9(b), 5.9(c) and 5.9(f) show instantaneous buffer sizes along with the corresponding quality level values achieved by SDQA,SDQA-AA (m= 5),AVIS-DP andAVIS-GREEDY strategies respectively, for a single flow (namely, Star Wars) over the entire simulation duration. The scenario considers a cell with 50 active flows. It may be observed from the figures that the AVSA based adap- tation strategies (i.e. SDQAand SDQA-AA) are able to maintain stable playout buffer sizes throughout their transmission duration. However, the AVIS based adaptation strategies (i.e. AVIS-DP and AVIS-GREEDY) encounter frequent
0 5 10 15 20 25 30
0 50 100 150 200 250 300 0 2 4 6 8 10
Buffer Size (Secs) Encoding Quality Level (QL)
Time (Secs) Playout Buffer Size (Secs)
Quality Level (QL)
(a) SDQA
0 5 10 15 20 25 30
0 50 100 150 200 250 300 0 2 4 6 8 10
Buffer Size (Secs) Encoding Quality Level (QL)
Time (Secs) Playout Buffer Size (Secs)
Quality Level (QL)
(b) SDQA-AA (m= 5)
0 5 10 15 20 25 30
0 50 100 150 200 250 300 0 2 4 6 8 10
Buffer Size (Secs) Encoding Quality Level (QL)
Time (Secs) Playout Buffer Size (Secs)
Quality Level (QL)
(c) AVIS-DP
0 0.5 1 1.5 2
0 50 100 150 200 250 300
Throttling Rate
Time (Secs)
(d) SDQA
0 0.5 1 1.5 2
0 50 100 150 200 250 300
Throttling Rate
Time (Secs)
(e) SDQA-AA(m= 5)
0 5 10 15 20 25 30
0 50 100 150 200 250 300 0 2 4 6 8 10
Buffer Size (Secs) Encoding Quality Level (QL)
Time (Secs) Playout Buffer Size (Secs)
Quality Level (QL)
(f) AVIS-GREEDY
Figure 5.9: Comparative results of performance metrics Vs. Time for SDQA, SDQA- AA (m= 5) and AVIS
re-buffering events because they are oblivious of instantaneous playout buffer sizes. From Figures 5.9(a) and 5.9(b) we see that initially (i.e. when the flow has just started), buffer size grows at a very fast rate. Such fast buffer growth rates during startup allows the system to achieve low startup delays, thus contributing to better overall QoE for all the adaptation schemes. Quick buffer ramp-ups for startup flows are mainly achieved by higher throttling rates (refer Figures 5.9(d) and 5.9(e)) during initial transmission phase of a flow. It may be seen that such a throttling rate control mechanism has been effective in maintaining stable buffer sizes for the flow over the entire simulation length. On the other hand, theAVIS based adaptation strategies are unaware of client-side buffer status and therefore, do not adjust throttling rates (the throttling rate remains constant at 1) during transmission. Figure 5.9(f) shows that being purely greedy in nature, AVIS-GREEDY delivers poorer average video quality levels ∼5.5 (which corre- sponds to average bit-rate about 27.5 KBps) compared toSDQA,SDQA-AAand AVIS-DP which deliver video qualities of about ∼ 6.5 (bit-rate ∼ 37.8 KBps).
5.3 Experiments and Results
0 0.2 0.4 0.6 0.8 1
10 20 30 40 50
—S—w—i—t—c—h—i—n—g
#Video Flows
SDQA SDQA-AA (m=5)
(a) Without stability moder- ation
0 0.2 0.4 0.6 0.8 1
10 20 30 40 50
—S—w—i—t—c—h—i—n—g
#Video Flows
SDQA SDQA-AA (m=5)
(b) With stability modera- tion;T h sw= 0.5
0 0.2 0.4 0.6 0.8 1
10 20 30 40 50
—S—w—i—t—c—h—i—n—g
#Video Flows
SDQA SDQA-AA (m=5)
(c) With stability modera- tion;T h sw= 0.25
30 35 40 45 50
10 20 30 40 50
—P—S—N—R
#Video Flows SDQA SDQA-AA (m=5)
(d) Without stability moder- ation
30 35 40 45 50
10 20 30 40 50
—P—S—N—R
#Video Flows SDQA SDQA-AA (m=5)
(e) With stability modera- tion;T h sw= 0.5
30 35 40 45 50
10 20 30 40 50
—P—S—N—R
#Video Flows SDQA SDQA-AA (m=5)
(f) With stability modera- tion;T h sw= 0.25
Figure 5.10: Results forSwitching andP SN R without stability moderation and with stability moderation (at T h sw= 0.5,0.25)
Additionally, it may be observed thatAVIS-GREEDY may be subject to harsher quality level fluctuations compared to the other schemes.