CHAPTER 5 RESULTS
5.5 INTERFERENTIAL STATISTICS
The factor analysis on the constructs of computer simulations (phase 3) also relates to each of the four engagement components (Table 5-15). The variations in communalities are small.
Consideration of the MSA values (0.80, 0.80, 0.85 0.77) which are all above 0.76, the validity of the different constructs on computer simulation was assured.
The results of sections 5.4.1 and 5.4.2 above showed that each construct of related sub-questions in the pre-test was reliable and valid. The constructs could thus be statistically processed and compared to each other, as reported in section 5.5.
According to the results displayed on Table 5-16 the mean scores for all the constructs in the pre- and post-tests on all the three strategies were above 3, leaning towards an average of 4, showing that the degree to which the teachers agreed with the statements is between the options Agree and Strongly agree.
Similar mean values were obtained in the pre- and post-tests for two context-based constructs, namely cognitive (P2Q1) and behaviour (P2Q3). For all other constructs, the mean values increased, i.e., the students agreed more strongly with the statements after the intervention. In order to determine the practical significance of the changes in means, Cohen effect sizes were determined.
Cohen effect size d
Cohen’s effect size d for dependent groups were used to compare the means of the constructs before and after the intervention and determine the significance thereof. The two matched groups were the pre-test and post-test results for each engagement component assessed for the different intervention strategies (music, context-based inquiry, computer simulation as engagement strategies). Since random sampling was not done, Cohen effect sizes (d) was used to indicate whether practical significance was achieved (Steyn & Ellis, 2003; Cohen, 1988). Practical significance is the degree to which the difference is big enough to have an impact in practice.
This includes interpretation of comparisons between the construct means, as illustrated in Table 5.17.
The interpretation of the d-values regarding mean differences in Table 5.17 is as follows: small effect: d = |0.2|; medium effect (noticeable with the naked eye): d = |0.5|; large effect (practically significant): d ≥ |0.8| (Cohen, 1988).
Table 5-17: Effect sizes for differences in pre-and post -test results Constructs N
Mean- of difference between pre-and
post-test
Std deviation of mean Difference
p-value (in case of random sampling)
d- value
MUSIC
Cognitive
P1Q1 44 0.319 0.619 0.001* 0.58 Δ
Affective
P1Q2 41 0.117 0.579 0.203 0.24
Behaviour
P1Q3 42 0.273 0.584 0.004* 0.51 Δ
Authentic
P1Q4 44 0.10 0.591 0.267 0.21
Constructs N
Mean- of difference between pre-and
post-test
Std deviation of mean Difference
p-value (in case of random sampling)
d- value
CONTEXT-BASED
Cognitive
P2Q1 36 -0.045 0.743 0.714 0.10
Affective
P2Q2 36 0.083 0.596 0.407 0.13
Behaviour
P2Q3 35 -0.011 0.682 0.922 0.02
Authentic
P2Q4 35 0.006 0.622 0.957 0.01
COMPUTER SIMULATION Cognitive
P3Q1 36 0.25 0.556 0.011* 0.51 Δ
Affective
P3Q2 35 0.134 0.428 0.072 0.29
Behaviour
P3Q3 35 0.109 0.532 0.236 0.24
Authentic
P3Q4 35 0.113 0.496 0.187 0.23
* Statistically significant at 0.01 level according to t-test results for dependent groups Δ Medium effect in practice
As seen from Table 5-17, there was a mean difference (0.319) with a medium effect size (d=0.58) between the pre- and post-test mean scores of the cognitive construct of music (P1Q1).
Meaning that the post-test mean score of 3.53 (Table 5-16) obtained for music on the construct cognitive was larger than the pre-test mean score 3.22 (Table 5-16) with a medium effect in practice and that the music intervention thus had a noticeable affect that can be seen with the naked eye on the perceptions of the teachers regarding cognitive engagement skills.
Furthermore, if random sampling was assumed, the music intervention could be regarded as statistically significant on a 0.001 level on cognitive engagement skills. Music behaviour produced similar results (refer to Tables 5-16 and 5-17). Therefore, the music intervention also had a noticeable effect with the naked eye on the perceptions of the teachers regarding behavioural engagement skills. Additionally, it is statistically significant at a value of 0.004.
With regard to computer simulations, the results in Table 5-17 show a mean difference of 0.25 with a medium effect size (d=0.51) between the pre- and post-test mean scores of the cognitive construct (P3Q1). This implies that the post-test mean score 3.49 (see Table 5-16) obtained for this construct was larger than the pre-test mean score 3.73 (see Table 5-16) with a medium effect in practice. Thus, the computer simulation intervention had a noticeable effect with the naked eye on the perceptions of the teachers regarding the cognitive skills. Furthermore, if random sampling
was assumed the computer simulation intervention could be regarded as statistically significant on a 0.01 level for the participants' perceptions on cognitive skills. The d-value (d=0.51) obtained for the cognitive construct of computer simulation indicates that this intervention had a medium effect on the perceptions of the teachers regarding cognitive engagement through computer simulation, while the effects on affective and behavioural were small (0.2 ≤ d ≤ 0.5).
Regarding all other constructs of music, context-based inquiry and computer simulation, there were no or small differences between pre and post-tests, meaning that the interventions did not influence the teachers’ perceptions on those other constructs, i.e., P1Q2 music affective; P1 Q2 and Q3 context-based inquiry; computer simulation behaviour.
In summary, the interventions had a medium effect that is noticeable with the naked eye on the following constructs (with 0.5 ≤ d ≤ 0.8): P1Q1 music cognitive, P1Q3 music behaviour, and P3Q1computer simulation cognitive. The constructs under context-based inquiry showed no significant difference between pre and post-test responses, with a consequent low d-value (d<0.2). On the other hand, computer simulations impacted positively with small to medium effect on all four engagement components.
The statistical analysis described in section 5.5 from the numeric quantitative data includes descriptive statistics like the mean and standard deviation. This led to the derivation of important facts from the research data such as MU afford cognitive and behaviour engagement while CS affords cognitive engagement.
In addition, in section 5.6, the qualitative analysis provided details, and gathered more in-depth information to build on the quantitative data collected to address the research questions. The natural responses from all the respondents (participants) and observational and recorded data from the focus group were also analysed and interpreted in this section.