4.6 Results and Discussions
4.6.2 Analysis of Fine Motor Skills with Quantitative Metrics
Several performance metrics were derived from the recorded data to analyze the fine motor skills of participants (Table 4-3).
Table 4-3. Performance Metrics.
Metrics Description
Grip Force Inside/Outside Path (Fin, Fout) The average values of the overall grip force the user applies inside the path or outside the path.
Grip Force Variation Inside/Outside Path (CVin, CVout)
The coefficient of variations of grip forces that are calculated as the ratio of the standard deviation to the average of Fin and Fout.
Position Error (PE) The root-mean-square error (RMSE) between the actual position of controlled virtual object and the optimal position in the letter path.
Movement Speed (V) The average value of the overall movement speeds the user generates during the letter path.
Speed Variation (CVv) The coefficient of variation of the overall movement speeds that are computed as the ratio of the standard deviation to the average of V.
Task Score (TS) The ratio of the final points (the difference between the total reward points and the total penalty points) to the maximum points.
Task Efficiency (TE) The final points divided by the completion time in one task.
To assess grip control, we measured the average grip force (F) the participants applied during the task as well as the grip force variation (CVF), which suggested the degree of variability in grip force. Because
the Letter Tasks were designed to require medium grip force to write the letters inside the letter path and allow to apply grip force freely outside the letter path, the participant was anticipated to apply different grip forces when moving inside the letter path than when outside the letter path. Thus, we evaluated the grip control in different cases (Fin, Fout, CVin, CVout).
As for the hand movement control, we measured the position error (PE), movement speed (V) and speed variation (CVv). The position error measured the position difference from the actual position to the optimal position in the letter path. Lower position error suggested that participants tended to move closer to the center of the paths and thus reduce the risk of hitting the walls. The movement speed indicated how fast the movement manipulation was performed, while the speed variation indicated the degree of variability in movement speed.
To evaluate the overall performance, we computed the task score (TS) and task efficiency (TE). During the tasks, the participants got reward points when completing a letter and lost points when hitting the wall.
The final points the participant got for one task was the difference between the total reward points and the total penalty points. The task score was defined as the final points divided by the maximum points the task provided. The task efficiency considered the completion time the participant used for one task and indicated the achieved points per minute.
We used Spearman’ rank correlation analysis to investigate the correlation between the proposed performance metrics and VMI Motor Coordination Test scores (VMI scores). Table 4-4 presents the correlations for all participants as well as for TD group and ASD group. The correlation directions were consistent in two groups. Significant correlations (p < .05) were only found between force-related data and VMI scores.
Table 4-4. Correlations between Performance Metrics and VMI Scores.
Metrics All TD Group ASD Group
ρ p ρ p ρ p
Fin (N) 0.488 .016* 0.618 .032* 0.352 .262 CVin -0.487 .016* -0.523 .081 -0.430 .163 Fout (N) 0.427 .038* 0.414 .181 0.345 .272 CVout -0.442 .031* -0.235 .462 -0.366 .242 PE(cm) -0.301 .153 -0.358 .253 -0.310 .327 V(cm/s) -0.148 .491 -0.239 .455 -0.232 .467 CVv 0.017 .937 0.284 .371 0.063 .845 TS 0.276 .192 0.368 .239 0.332 .292
*p<.05
Since normality of this small sample (N = 6 for each group) could not be assumed, we used the Wilcoxon
signed-rank test to compare participant pre- and post-test performance, and used the Wilcoxon rank-sum test to explore the group differences [71]. We report r effect sizes with the significance cutoffs of large (>0.5), medium(>0.3) and small (>0.1) [72]. Table 4-5 presents the means and standard deviations, the percentage of change from pre-test to post-test, and the test results.
Table 4-5. Performance Results of TD Group (N = 6) and ASD group (N = 6)
Metrics
TD Group ASD Group
Pre Post
RC
(%) Z p r
Pre Post
RC
(%) Z p r
Mean (SD)
Mean (SD)
Mean (SD)
Mean (SD)
Fin (N) 3.63 (0.41)
3.69
(0.35) 1.65 -0.52 .600 -.15 3.34 (0.294)
3.84
(0.31) 15.0 -1.99 .046* -.58 CVin
0.31 (0.08)
0.25
(0.06) -19.4 2.20 .028* .64 0.35 (0.18)
0.26
(0.11) -25.7 1.78 .075 .51 Fout (N) 1.78
(0.44)
2.01
(0.32) 12.9 -1.99 .046* -.58 1.60 (0.66)
1.88
(0.43) 17.5 -1.15 .249 -.33 CVout
0.89 (0.34)
0.67
(0.13) -24.7 1.99 .046* .58 0.98 (0.37)
0.83
(0.27) -15.3 1.99 .046* .58 PE(cm) 0.37
(0.04)
0.36
(0.05) -2.7 0.52 .600 .15 0.39 (0.08)
0.37
(0.05) -5.13 1.15 .249 .33 V(cm/s) 4.15
(1.17)
4.22
(0.50) 1.69 -0.31 .753 -.09 4.31 (1.31)
4.78
(0.96) 10.9 -1.78 .075 -.51 CVv
1.48 (0.09)
1.48
(0.15) 0 -0.52 .600 -.15 1.62 (0.38)
1.39
(0.08) -14.2 2.20 .028* .64
TS 0.73
(0.13)
0.80
(0.11) 9.59 -0.94 .345 -.27 0.53 (0.73)
0.74
(0.29) 39.6 -0.94 .345 -.27 TE
(/minute)
21.38 (3.96)
27.20
(5.54) 28.2 -1.99 .046* -.58 17.54 (18.25)
27.74
(7.47) 58.2 -2.2 .028* -.64
VMI 94
(4)
105
(6) 11.7 -2.21 .027* -.64 89 (11)
102
(11) 14.6 -2.2 .028* -.64 VMI, scores on the VMI Motor Coordination test
RC, relative change computed by (post - pre)/pre *100%
*p<.05
4.6.2.1 Grip Control
As participants executed virtual pen strokes within the outlines of the letters, they were required to apply forces (Fin) within 2.96-5.50N in order to get reward points. The correlation analysis indicated that Fin was positively correlated to the VMI scores in both TD and ASD group. Significant correlation was found in
TD group (ρ = 0.618, p = .032), while medium correlation was found in ASD group (ρ = 0.352, p = .262).
The correlations suggested increasing force within an allowable range might contribute to higher VMI scores. The performance analysis indicated that compared to the TD group who applied similar forces in both pre- and post-test (TD: RC = 1.65%, Z = -0.52, p = .600, r = -.15), the ASD group significantly increased the applied force in post-test with a large effect size (ASD: RC = 15%, Z = -1.99, p = .046, r = - .58). In the pre-test, the ASD group applied smaller forces than the TD group. Though no significant difference was detected, a medium effect size existed between the TD group and ASD in the pre-test (Z = 1.20, p = .229, r = .35). Force data suggested that in the pre-test, participants in the TD group were able to quickly find and steadily maintain proper forces, while most in the ASD group tended to apply small forces closer to the lower limit of valid force range. However, the ASD participants gradually increased the applied forces, and after the practice session, the ASD group applied much larger forces in the post-test. In addition, the Force Variation Inside Letter (CVin) was negatively correlated to the VMI scores. Large correlation was found in TD group (ρ = -0.523, p = .081), while medium correlation was found in ASD group (ρ = 0.430, p = .163). The correlations suggested that lower variability in grip force might lead to higher VMI scores.
The performance analysis found that CVin significantly dropped in the TD group with a large effect size (TD: RC = -19.4%, Z = 2.20, p = .028, r = .64), and also decreased in the ASD group with a large effect size (ASD: RC = -25.7%, Z = 1.78, p = .075, r = .51). The results implied that both groups were able to maintain stable grip force while writing after practicing fine motor tasks in the practice session.
Outside the letters, participants were allowed to freely grip the Haptic Gripper. Results showed that both groups applied smaller grip forces (Fout < 2.96N) outside the letters, which indicated that there were significant differences in grip force between when one was writing (Fin) and when one was simply holding the gripper (Fout) (TD: Z = 2.20, p = .028, r = .64; ASD: Z = 2.20, p = .028, r = .64). Simultaneously, the Spearman’s rank correlation analysis indicated large positive relationships between Fin and Fout (TD: ρ = 0.714, p = .111; ASD: ρ = 0.543, p = .266), which suggested the consistency of grip forces applied by the participants. As both groups increased Fin in the post-test, we also found that both groups increased Fout in the post-test (TD: RC = 12.9%, Z = -1.99, p = .046, r = -.58; ASD: RC = 17.5%, Z = -1.15, p = .249, r = - .33). In addition, the correlation analysis also suggested that smaller CVout might help improve the VMI scores (All: ρ = -0.442, p = .031). Both groups achieved significant decreases in force variations outside the letter (CVout) with large effect sizes (TD: RC = -24.7%, Z = 1.99, p = .046, r = .58; ASD: RC = -15.3%, Z = 1.99, p = .046, r = .58), which suggests that both groups reduced the force variability outside the letters.
Though no requirements were set for how the user should grip when he/she was not writing, the grip strategy and variability seemed also important that could affect the performance.
4.6.2.2 Hand Movement Control
The correlation analysis indicated the medium negative relationships between the position errors (PE) and the VMI scores (TD: ρ = -0.358, p = .253; ASD: ρ = -0.310, p = .327). Significant negative relationships between the position errors (PE) and the virtual task scores (TS) were also found (TD: ρ = - 0.832, p <.001; ASD: ρ = -0.883, p < .001). It was reasonable to expect that moving closer to the optimal path could achieve higher scores. Though no statistically significant differences were found in either the TD or ASD group regarding the position error (PE), both group decreased the average position errors in the post-test (TD: RC = -2.70%, Z = 0.52, p = .600, r = .15; ASD: RC = -5.13%, Z = 1.15, p = .249, r = .33).
The correlation analysis indicated that only small negative relationships were found between the movement speed (V) and the VMI scores (TD: ρ = -0.239, p = .455; ASD: ρ = -0.232, p = .467), and small positive relationships between the speed variation (CVv) and the VMI scores (TD: ρ = 0.284, p = .371;
ASD: ρ = 0.063, p = .845). These results suggested that the movement speed did not have much influence on the performance of the VMI Motor Coordination Test. However, significant negative relationships were found between the movement speed (V) and virtual task scores (TS) (TD: ρ = -0.601, p = .039; ASD: ρ = - 0.592, p = .043), suggesting that the movement speed was an important factor affecting the task performance. The performance analysis showed that in either pre-test or post-test, the ASD group tended to move faster than the TD group on average. Especially in the post-test, while the TD group maintained the similar movement speed (RC = 1.69%, Z = -0.31, p = .753, r = -.09) and similar speed variations (RC
= 0%, Z = -0.52, p = .600, r = -.15), the ASD group showed increase in movement speed (RC = 10.9%, Z
= -1.78, p = .075, r = -.51) and significant decrease in speed variation (RC = -14.2%, Z = 2.20, p = .028, r
= .64), which might suggest that the ASD group moved more smoothly in the post-test even when increasing the movement speed.
4.6.2.3 Performance Improvement
Scores on the VMI Motor Coordination test (VMI) indicated that participants in both TD group and ASD group achieved statistically significant improvements on the post-test, with large effect sizes (TD: RC = 11.7%, Z = -2.21, p = .027, r = -.64; ASD: RC = 14.6%, Z = -2.20, p = .028, r = -.64). According to the VMI standard score interpretation [23], the performance of the TD group remained at the average level (90- 109) in both pre- and post-test, while the ASD group stayed at the below-average level (80-89) in the pre- test and reached the average level in the post-test.
Unlike the VMI Motor Coordination test that was completed within exactly 5 minutes, the virtual fine motor tasks were ended when the user achieve the task goals. Thus, except for the task score (TS), we also used the task efficiency (TE) that considered the time variable as the task performance metric. The performance analysis indicated significant improvements with large effect sizes (TD: RC = 28.2%, Z = -
1.99, p = .046, r = -.58; ASD: RC = 58.2%, Z = -2.20, p = .028, r = -.64) in both groups. Typically, the ASD group achieved significant improvements in all three tasks with large effect sizes (“THE” task: RC = 35.7%, Z = -2.20, p = .028, r = -.64; “LAZY” task: RC = 77.3%, Z = -2.20, p = .028, r = -.64; “DOG” task:
RC = 56.3%, Z = -1.99, p = .046, r = -.58). The improved raw task scores (TS) also existed in both groups, but no statistically significant differences between pre- and post-test were found in either the TD or ASD group (TD: RC = 9.59%, Z = -0.94, p = .345, r = -.27; ASD: RC = 39.6%, Z = -0.94, p = .345, r = -.27).
The performance improvements in both VMI test and virtual test suggested that the virtual practice tasks (Path Tasks) had positive impact on improving the finger and hand motor control. Medium positive relationships between the virtual task scores (TS) and the VMI scores (TD: ρ = 0.368, p = .239; ASD: ρ = 0.332, p = .292) were found, which also suggested the effectiveness of the tasks in our system to evaluate the user’s fine motor abilities.