Chapter 5 Data Analysis of the Study on User Adaptivity
5.3. Computing User Adaptivity
ππ¬ππ« ππππ©ππ’π―π’ππ² πππ¨π«π (πππ) = βπβπππ’=πβπ’ (5.7)
User adaptivity therefore measures the userβs degree of expertise achieved in familiarizing to an interface.
We conducted total 5 trials of adaptivity tests on the participants for GUI-1 and GUI-2.
The participants became familiarized with the interfaces while solving the learning tests.
The participantβs performance on adaptivity tests and computation of user adaptivity scores for GUI-1 and GUI-2 for all the three sample groups (K, C and A) are presented in the next section.
Figure 5.2 Performance curve of Pt and Pc for GUI-1 in group (K)
Figure 5.3 Performance curve of Pt and Pc for GUI-2 in group (K) 0.477
0.636
0.708 0.687 0.720
0.765 0.745 0.752 0.786 0.768
0.000 0.200 0.400 0.600 0.800 1.000 1.200
T1 T2 T3 T4 T5
Trial
Performence Index
Performance Curve - GUI-1 Group(K)
Pt Pc
0.266
0.441
0.506 0.553 0.548
0.879
0.969 1.000 1.040 1.029
0.000 0.200 0.400 0.600 0.800 1.000 1.200
T1 T2 T3 T4 T5
Trial
Performence Index
Performance Curve - GUI-2 Group(K)
Pt Pc
Figure 5.4 Performance curve of Pt and Pc for GUI-1 in group (C)
Figure 5.5 Performance curve of Pt and Pc for GUI-2 in group (C) 0.460
0.703
0.775 0.792
0.809
0.731 0.747 0.750 0.794
0.754
0.000 0.200 0.400 0.600 0.800 1.000 1.200
T1 T2 T3 T4 T5
Trial
Performence Index
Performance Curve - GUI-1 Group(C)
Pt Pc
0.213
0.399
0.461
0.525 0.548
0.815
0.947 0.972 1.020 1.022
0.000 0.200 0.400 0.600 0.800 1.000 1.200
T1 T2 T3 T4 T5
Trial
Performence Index
Performance Curve - GUI-2 Group(C)
Pt Pc
Figure 5.6 Performance curve of Pt and Pc for GUI-1 in group (A)
Figure 5.7 Performance curve of Pt and Pc for GUI-2 in group (A)
0.440
0.633 0.693 0.700 0.732
0.767
0.722 0.731 0.773
0.728
0.000 0.200 0.400 0.600 0.800 1.000 1.200
T1 T2 T3 T4 T5
Trial
Performence Index
Performance Curve - GUI-1 Group(A)
Pt Pc
0.253
0.430
0.502 0.563 0.557
0.899
1.006 1.022 1.079
1.038
0.000 0.200 0.400 0.600 0.800 1.000 1.200
T1 T2 T3 T4 T5
Trial
Performence Index
Performance Curve - GUI-2 Group(A)
Pt Pc
Analysis of Effort Performance Index Curves (Pc)
Figures 5.2-5.7 show that performance index on GUI-2 for the 5th (last) trial of adaptivity test is 1.0287, 1.0225 and 1.0382 for group K, C and A respectively as against 0.768, 0.7539 and 0.7282 on GUI-1 in group K, C and A respectively. We may therefore conclude that participants gain higher degree of expertise in interaction effort on GUI-2 as compared to GUI-1. The shape of the Pc curve for GUI-2 shows a steady rise in each trial. This reflects accumulation of performance benefits or knowledge of the interface and therefore indicates an adapting behavior of the participant. Against this in case of GUI-1, performance benefits are being lost due to ups and downs in the shape of the Pc curve which shows a non-adapting behavior of the participant.
We may conclude that participants need more practice to complete tasks in minimum efforts using hierarchical horizontal menu as compared to non-hierarchical split menu interface.
Analysis of Time Performance Index Curves (Pt)
We observe that in time performance (Pt), participants using GUI-1 perform better compared to participants using GUI-2. Time performance index in 5th (last) trial is 0.7201, 0.8091 and 0.7317 for GUI-1 in group K, C and A respectively as compared to 0.5483, 0.5479 and 0.5571 for GUI-2 in group K, C and A respectively. For both the GUIs the shape of the Pt curve shows a steady rise in successive trials indicating adapting behavior of the participants.
It may be noted that userβs need more practice to increase their speed of tasks using non- hierarchical split menu interface as compared to hierarchical horizontal menu interface.
We computed integrated performance index of ith trial Pi, as the product of effort performance index PCi and time performance index Pti (Eqn. 5.3). Figures 5.8-5.13 show average integrated performance index Pi of the group against trial, for GUI-1 and GUI-2 in sample groups K, C and A respectively. In figure 5.8, the X axis shows the trial number T1 to T5 and Y axis shows integrated performance index Pi. Same is repeated for figures 5.9-5.13.
Figure 5.8 Performance curve of Pi for GUI-1 in group (K)
Figure 5.9 Performance curve of Pi for GUI-2 in group (K) 0.374
0.478
0.536 0.547 0.559
0.000 0.200 0.400 0.600 0.800 1.000
T1 T2 T3 T4 T5
Trial
Performence Index (Pi)
Performance Curve - GUI-1 Group(K)
Pi
0.246
0.435
0.512
0.579 0.567
0.000 0.200 0.400 0.600 0.800 1.000
T1 T2 T3 T4 T5
Trial
Performence Index (Pi)
Performance Curve - GUI-2 Group(K)
Pi
Figure 5.10 Performance curve of Pi for GUI-1 in group (C)
Figure 5.11 Performance curve of Pi for GUI-2 in group (C) 0.348
0.532
0.588 0.639 0.617
0.000 0.200 0.400 0.600 0.800 1.000
T1 T2 T3 T4 T5
Trial
Performence Index (Pi)
Performance Curve - GUI-1 Group(C)
Pi
0.185
0.391
0.456
0.545 0.570
0.000 0.200 0.400 0.600 0.800 1.000
T1 T2 T3 T4 T5
Trial
Performence Index (Pi)
Performance Curve - GUI-2 Group(C)
Pi
Figure 5.12 Performance curve of Pi for GUI-1 in group (A)
Figure 5.13 Performance curves of Pi for GUI-2 in group (A) 0.341
0.464
0.514 0.551 0.537
0.000 0.200 0.400 0.600 0.800 1.000
T1 T2 T3 T4 T5
Trial
Performence Index (Pi) Performance Curve - GUI-1 Group(A)
Pi
0.236
0.438
0.518
0.613
0.585
0.000 0.200 0.400 0.600 0.800 1.000
T1 T2 T3 T4 T5
Trial
Performence Index (Pi)
Performance Curve - GUI-2 Group(A)
Pi
Analysis of Integrated Performance Index Curves (Pi)
Figures 5.8-5.13 show that after 5 trials, participants using GUI-1 and GUI-2 both attain similar degrees of expertise in adaptivity test, in all the three sample groups K, C and A.
The performance index of the 5th (last) trial for GUI-1 is 0.5591, 0.6174 and 0.5367 for group K, C and A respectively and for GUI-2 it is 0.5673, 0.5698 and 0.5848 respectively. However, we observe that the performance improvement rate on GUI-2 is greater as compared to GUI-1 indicating a higher capability of adaptivity.
From the graphs of Pi it can be concluded that userβs adaptivity is similar for hierarchical horizontal menu interface and non-hierarchical split menu interface.
We used PHP program to compute the positive, neutral and negative benefits and corresponding performance gains in each trial of adaptivity test for each individual using Eqns. (5.4), (5.5) and (5.6). Total benefits accumulated by all participants of the group during the adaptation period (5 trials) on GUI-1 and GUI-2 in sample groups K, C and A are shown in table 5.1.
Table 5.1 Performance benefits in adaptation period Performance Benefits
In 5 Trials User Interface
Number of negative benefits
Number of positive benefits
Performance
Accumulated Mean Adaptivity Group-K
(N=85)
GUI-1 127 213 105.36 0.3099
GUI-2 102 238 111.05 0.3266
Group-C (N=106)
GUI-1 139 285 155.1 0.3658
GUI-2 109 315 134.49 0.3172
Group-A (N=96)
GUI-1 133 251 120.9226 0.3149
GUI-2 106 278 133.0456 0.3465
The first column of table 5.1 represents group with sample size. The second column is the user interface, followed by the number of negative benefits and then positive benefits.
Total performance accumulated by the entire group is shown next and average user adaptivity score of the group in represented in the last column.
Data of table 5.1 indicates that, users using GUI-2 accumulate less negative benefits and more positive benefits compared to GUI-1. However, the performance gain accumulated from these benefits is almost same for GUI-1 and GUI-2. This is because in complex interfaces learning is difficult (performance gains are low) and forgetting is easy (performance losses are heavy), while in simpler interfaces learning is easy (performance gains are high) and forgetting is difficult (performance losses are low).
Adaptivity tests show that consistency in performance improvement is better in non- hierarchical split menu interface compared to hierarchical horizontal menu interface.
We normalized the performance benefit values in table 5.1 with the sample size. The normalized values are shown in figure 5.14 for all the three sample groups.
Figure 5.14 Normalized performance benefits across GUIs in adaptation period
Figure 5.14 shows that, on GUI-1, participants of group-C accumulate higher benefits and gain more expertise of interface as compared to the other learning groups. We do not observe this difference in groups on GUI-2.
1.49
1.20
2.51
2.80
1.24
1.31
0.31 0.33
1.31
1.03
2.69
2.97
1.46
1.27
0.37 0.32
1.39
1.10
2.61
2.90
1.26
1.39
0.31 0.35
0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50
GUI-1 GUI-2 GUI-1 GUI-2 GUI-1 GUI-2 GUI-1 GUI-2
Negative Benefits Positive Benefits Accumlated Performance
Mean Adaptivity Group(K) Group(C) Group(A)
This implies that the type of learning tasks which are practiced on simple hierarchical horizontal menu interface influences adaptivity, however for complex non-hierarchical split menu interface the learning task type which is practiced does not influence adaptivity.
We computed user adaptivity scores (UAS) using Eqn. (5.7) for GUI-1 and GUI-2 of each participant. Groupβs mean UAS in percentage for both GUIs are shown in figure 5.15.
Figure 5.15 Group mean of user adaptivity scores (UAS)
Figure 5.15 indicates an overall adaptivity of 30%-35% is attained in 5 trials. We also observe that participants have similar adaptivity scores on GUI-1 and GUI-2.
We observe that on hierarchical horizontal menu interface, userβs adaptivity is influenced by complexity of the learning tasks executed on it. Adaptivity on non-hierarchical split menu interface is insensitive to the complexity of learning tasks carried out on them.
30.99%
36.58%
31.49%
32.66% 31.72%
34.65%
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
45.00%
50.00%
Group-K Group-C Group-A
Percentage Adaptivity
UAS_GUI-1 UAS_GUI-2