4.5 Weighted Voting based Attention Prediction on Images
4.5.2 Prediction Model
4.5. WEIGHTED VOTING BASED ATTENTION PREDICTION ON IMAGES
537
327
205 151
120129 10393 95
80 75 71 59 53
39 3223 16 5 6 2 1 1 median Fixation Index=4
0 200 400
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Fixation Index
Elements
(a) Frequency distribution
100 92.16
80.39 69.61
59.8 48.53
37.25 27.45
20.1 10.78
6.37 1.96 0
1 2 3 4 5 6 7 8 9 10 11 12 13
Minimum number of participants
Fixated elements (%)
(b) fixated images (%)
0 5 10 15 20 25 30
5 6 7 8 9 10 11 12 13
Number of Informative Visual Features
Saliency Threshold, θ
Uniform Weighting Linear Weighting Proportional Weighting Inverse Proportional Weighting
(c) Informative features
0 0.5 1 1.5 2 2.5 3
5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Entropy (in nats)
Saliency Threshold, θ
Uniform Weighting Linear Weighting Proportional Weighting Inverse Proportional Weighting
(d) Entropy
Figure 4.13: θ based variation in(c)number of information visual features, and(d) entropy, for four weighting strategies
Feature Selection
Feature selection helps to prune the redundant features and to select the informative features in explaining the effective visual attention. We considered the features with positive information gain scores for the analysis, analogous to [19]. The number of informative visual features are 24, 21, 24, 27 respectively for uniform, linear, proportional and inverse proportional weightings. Among them, 18 visual features were informative across the four weighting strategies. The informative features along with information gain scores are shown in Table 4.6.
Across the weighting strategies, the element’s ‘top position’ (distance from the top edge of the webpage) obtained the predominant information gain highlighting its attention explaining ability. The HISTOGRAM features succeeded the POSITION feature and are highly informative across the weighting strategies. However, the HISTOGRAM features corresponding to the pure colors (first and eighth histogram bins) were not found to be informative across all weighting strategies. This is attributed to their wide usage as
4.5. WEIGHTED VOTING BASED ATTENTION PREDICTION ON IMAGES Table 4.6: Information-gain scores of visual features at θ= 5 for four weighting strategies.
Color intensity represents the relative importance of a feature. The feature names starting with “hist” and “diff_hist” respectively denote the histogram features and contrast histogram features where the corresponding color component (R,G, B, gray) and bin number (1, . . . ,8) are suffixed. The “rect.top”, “rect.left”, “rect.bottom”, “size” denote the rectangular image element’s ‘top distance’, ‘left distance’, ‘bottom distance’, and ‘area’.
Group Feature Uniform Linear Proportional Inv. Pro.
POSITION rect.top 0.1110 0.1380 0.0976 0.1186
HISTOGRAM hist_R_6 0.1118 0.0998 0.0935 0.1043
HISTOGRAM hist_gray_7 0.1086 0.0809 0.0981 0.1174
HISTOGRAM hist_gray_5 0.1047 0.0876 0.1061 0.1217
HISTOGRAM hist_G_6 0.1037 0.0998 0.0967 0.1084
HISTOGRAM hist_G_5 0.1002 0.0773 0.0889 0.1101
HISTOGRAM hist_gray_6 0.0997 0.1054 0.1067 0.0989
POSITION size 0.0946 0.0967 0.0972 0.1193
HISTOGRAM hist_B_5 0.0982 0.0799 0.0852 0.1030
HISTOGRAM hist_R_5 0.0925 0.0842 0.0947 0.1103
HISTOGRAM hist_R_4 0.0802 0.1389 0.0671 0.0962
HISTOGRAM hist_B_3 0.0863 0.0869 0.0781 0.0851
HISTOGRAM hist_B_6 0.0877 0.0707 0.0665 0.0816
HISTOGRAM hist_B_4 0.0763 0.0692 0.0700 0.0985
POSITION rect.left 0.0746 0.1014 0.0635 0.0919
HISTOGRAM hist_gray_3 0.0720 0.0773 0.0675 0.0921
HISTOGRAM hist_gray_4 0.0664 0.0710 0.0619 0.0858
HISTOGRAM hist_B_2 0.0660 0.0743 0.0692 0.0916
HISTOGRAM hist_R_7 0.0931 0.0775 0.1119
HISTOGRAM hist_G_7 0.0887 0.0810 0.0980
HISTOGRAM hist_G_3 0.0795 0.0617 0.0936
CONTRAST diff_hist_G_5 0.0743 0.0776 0.0927
HISTOGRAM hist_G_2 0.0695 0.0896
HISTOGRAM hist_G_4 0.0679 0.0794
CONTRAST diff_hist_R_3 0.0749 0.0614
HISTOGRAM hist_R_3 0.0684 0.0755
CONTRAST diff_hist_B_2 0.0527 0.0549
POSITION rect.bottom 0.0858
HISTOGRAM hist_gray_2 0.0799
background color (typically, white) across all the webpages, normalizing their attention explaining ability. Overall, the element’s position (‘top distance’, ‘left distance’), size and the de-saturated color histograms were found to be informative across all the weighting strategies. This signifies the prominence of image position, size, and intrinsic histogram visual features in explaining the free-viewing user attention.
Surprisingly, no COMPREHENSIVE feature was found to be informative in any weighting strategy and no CONTRAST feature (fromTable 3.3) was found to be informative across all four weighting strategies. Though three CONTRAST features were informative in at least one weighting strategy, the associated information gain scores were lower. The non- prominence of CONTRAST features may be attributed to the user’s inherent preference for the image elements on a webpage.
Number of Informative Features Vs. θ
To further understand the informative features, the number of informative visual features are computed with variation in θ as shown in Figure 4.13c. The number of informative features for uniform, linear, and inverse proportional weighting significantly decreased with an increase in θ. On the contrary, the number of informative features were relatively stable for proportional weighting up to θ = 9 after which it followed the reduction pattern of remaining weighting strategies. Beyond θ= 12, the number of informative visual features are approaching zero as other factors influence the attention which cannot be described by visual features.
Entropy Vs. θ
To understand the influence of θ on effectiveFI assignment (ground-truth), the entropy was computed for the four weighting strategies as shown in Figure 4.13d. At median θ, the linear weighting achieved the highest entropy among all strategies, and consistently remained till θ= 11. However, the increment in the entropy gradually reduced as the latter FIs are less frequent and were further weighted lower. The inverse proportional weighting achieved the second highest entropy till θ= 11 but outperformed the linear weighting beyond. The highest entropy achieved for the latter θis attributed to the higher weight assignment for the less frequent latter FIs. The uniform weighting and proportional weighting followed a similar entropy pattern, however, the latter weighting strategy achieved relatively lesser entropy owing to the FI’s frequency based weighting.
Observing the variation in number of informative visual features (Figure 4.13c) and the entropy (Figure 4.13d), we conclude that the distribution of effective fixation-indices varying significantly (higher entropy with increase inθ) which can not be explained by the visual features (lesser number of informative features with increase in θ). This also indicates, among the four strategies, proportional weighting strategy is a better candidate for modeling the effective visual attention with image visual features. However, to quantify and compare the prediction performance, metrics were computed for all the four strategies.