Video packet priority assignment based on spatio-temporal perceptual importance

The effectiveness of the proposed algorithm (both stages) is demonstrated using an intelligent packet drop application where it is compared with cumulative mean squared error (cMSE) based priority assignment and random packet dropping. The proposed algorithm does not require any prior training with subjective scores, which makes it easier to implement and deploy.

Application Layer Oriented Perspective of the Problem

It has been shown that visual interest pooling of the local SSIM indices significantly improves the performance [16]. Furthermore, the SSIM index has been shown to be effective in quantifying the effects of packet loss [10, 9].

Cross-Layer Oriented Perspective of the Problem

21] present a new cross-layer optimization approach that includes utility maximization where utility is defined as the temporal change in video quality. 25] propose a cross-layer solution at the MAC layer to find the optimal fragment size for each of the four priority classes assigned to H.264 slices.

Figure 1.1: Conceptual Framework of Cross Layer Optimization [27]

Deﬁnition and Need for Cross Layer Optimization

Deﬁnition

The lower layer serves its immediate upper layer, and the interfaces between the layers are strictly defined. So, although the upper layer receives services from the lower layer, it is unaware of the implementations and protocols of the lower layer.

Need for Cross Layer Optimization

Cross Layer Optimization Problem

Challenges in solving a Cross Layer Optimization Problem

MAC supports uneven fault protection for different flows or delay awareness, unlike traditional 802.11a/b/g MAC) which must be taken into account when performing the cross-layer optimization.

Classiﬁcation Of Cross Layer Solutions

Physical Layer Centric Strategy(A Bottom-up Approach)

In fact, the exchange of information between nearby base stations already dominates LTE systems. The Radio Resource Management (RRM) engine present at each base station is responsible for allocating resources in terms of bandwidth, power and time for each user. Thus, the required power scaling can be handled in a similar way through an eventual QoE specification that the network makes available to the RRM.

Well-known MAC Centric Approaches

Once the QoE status is known to the network, relevant information can be shared between base stations via the backhaul or via dedicated overhead channels. For example, real-time video traffic would have a QCI value in the range of 1 to 3, while for best data it would be in the range of 7 to 9. The weight for good high-priority traffic is calculated as the ratio of the average cMSE of slices with high priority and the average cMSE of all slices in the pre-encoded video, and similarly, the weight of a low-priority good transmission is calculated.

The goal is to find the optimal fragment sizes for low-priority and high-priority fragments so that weighted goodput is maximized. The number of high-priority slices generated per second is assumed to follow a uniform distribution over [0,N], where N is the total number of slices generated per second.

A Few Application Layer Centric Strategies(A Top-down Approach)

Packet Scheduling

If the receiver buffer has a shorter queue length than β, then the VPs to be sent will have a smaller D(∗) than their deadline thresholds, and as a result the importance criterion will be applied to them. On the other hand, if the receiver's buffer has a larger queue length than β, then D(∗) of the VPs to be sent is large enough and the VPs are sent consecutively. Frame-based planning is extended by adding motion texture discrimination. MPEG-4 supports data partitioning mode by separating the motion and texture by motion markers inserted between mo-.

If the texture information is lost, this approach uses the motion information to hide errors. Using this function rearranges the data in each frame with motion and texture blocks; all motion vector fields are collected in the motion block, and all DCT coeﬃcient fields in the texture block.

Cross Layer Perceptual ARQ Algorithm

For frameFi, the motion block is then divided into video packets denoted by V Pi,j(m)s and the texture block by V Pi,k(t)s. As it is clear, frame-based scheduling performs better than EDF and motion texture-based scheduling performs better than EDF and frame-based scheduling in terms of end-user video quality. The time instant of the first retransmission opportunity is set to be between the time instant of the first packet of the smallest frame interval and the last packet of the previous frame.

Di,n is the distortion impact given by cMSE, w is weight used to control the relative importance of the perceptual and temporal terms, K is the product of mean distortion and receiver buffer length and ∆t,n is the distance from deadline . As is clear, this ARQ scheduling outperforms standard 802.11 MAC-Layer ARQ in terms of video quality at the end user.

Video Packet Prioritization

GLM

A model is trained on a portion of the data (the training set) and then tested using the remaining data points (the test set). Once the coefficients are obtained, given a test video packet, its features are extracted and the inner product of the feature vector with the coefficient vector gives the value of the logit function from which the packet loss visibility ρ can be obtained. Some application layer techniques [2, 3, 4] rely on subjective assessment of packet loss effects to train linear model weights and select thresholds.

To our knowledge, this is the first application of PTQM in a multimedia communication framework. The efficiency of the proposed method is demonstrated by comparing it with existing prioritization techniques using a packet loss experiment that measures the perceptual quality of the received degraded video. The problem is formulated in Section 3.2 and the proposed algorithm is presented in Section 3.3.

Problem Statement and Assumptions

All parameters in this use previously determined values (eg in PTQM) and work well in the current setting.

Proposed Algorithm

Stage 1: Temporal Perceptual Importance

The methodology behind the threshold selection is inspired by the perceptual temporal quality metric (PTQM) [34]. The TQF weight is normalized by a factor that depends on the video frame rate. The temporal quality of the entire sequence is given by averaging the quality scores of all scenes.

The average motion content of the packet is given by the average of the motion vector magnitudes of all macroblocks in that packet. The threshold value of 4 was chosen empirically as it was found that a linear map of the mean motion vectors of all frames of a video (for several test videos) in the interval [0,10] resulted in averaging the mean motion vectors of all frames to be approximately 4.

Stage 2: Spatial Perceptual Importance

For each packet in the video, its TFBM value Tk is calculated and compared to the thresholdτt. It should be noted that a video decoded without induced packet errors is used as a "reference" when calculating the SSIM index. Then, the spatial importance Sk of the kth frame is calculated using the importance-weighted SSIM index defined in (3.4).

Threshold selection is the prioritization of the importance-weighted average SSIM score for the entire video. In this case, our algorithm gives the spatio-temporal meaning of that frame segment present in the video packet, and that packet is prioritized accordingly.

Results and Discussion

Experiments
Dataset
Results
Discussion

Moreover, this threshold results in roughly similar packet priority histograms to the proposed algorithm for most of the test videos. Table 3.1 shows that at a PLR of 5%, both Stage-1 and Stage-2 of the proposed algorithm outperform other policies for most videos. Furthermore, Stage-1 of the proposed algorithm easily outperforms the random packet drop policy of both PLRs.

From Table 3.3, it is clear that both stages of the proposed algorithm perform better than the other omission policies. As shown by the tables and plots in the previous section, the proposed algorithm outperforms the random drop policy and cMSE-based policy for a majority of the videos.

Figure 3.1: Network topology for the constrained bandwidth experiment.

Areas of Application

We would also like to note that several of the interesting observations and pitfalls noted by Chang et. It has been noted that the distance between lost frames in the case of double loss plays an important role in the visibility of the error. Furthermore, the performance of the proposed algorithm highlights the fact that perceptually motivated packet prioritization is a promising approach to estimate the perceptual effects of packet loss.

If the packets from different layers of the scalable encoded video are assigned priority using our algorithm, instead of dropping the entire enhancement layer, only low-priority packets from the enhancement layer can be dropped under low bitrate channel conditions. Thus, our technique can be used for packet priority assignment in the application layer in the existing cross-layer techniques to improve their performance.

Extension

Many cross-layer techniques lack an efficient prioritization technique at the application layer and use cMSE which in our work is proven to be less efficient compared to our technique. But features should not be linearly related to packet loss visibility, and features are interdependent. Construct a training matrix of size MxN with feature vectors as columns where M is the size of each feature vector and N is the number of training samples.

Use a suitable learning algorithm to classify the de-correlated feature vectors into required number of priority groups. The labels for the groups in supervised learning are obtained by subjective evaluation of the loss-induced video as mentioned in Section 2.9.1.

Two Queues Methodology

This scenario takes into account customer preferences regarding rebuffering and costs. For example, a video client may be willing to tolerate rebuffering in exchange for higher average quality (for example, to watch a movie in HD over a poor network) and may want to trade off QoE against delivery costs. In a multi-user environment, individual optimization strategies must consider the impact on other users and the optimization must be foresighted. That is, optimizing video quality in the short term without considering the effect of the current decision on long-term quality is not possible. a good methodology. In the first stage, the impact of packet loss on temporal quality was estimated, while in the second stage, the effect of packet loss on spatial quality was estimated.

The performance of the proposed method over the cMSE-based prioritization method was demonstrated using an intelligent packet-dropping application. Since multi-layer optimization is not optimal and does not meet the goal of maximizing end-user QoE, cross-layer optimization is used in multimedia traffic management.