4.1 The Computer Spreadsheet Exercises
4.1.8 Spreadsheet Assignment 2 - Non-flow and Steady-flow Energy Problems and
89 Table 4.6(b): Predicted and actual numbers of groups in normal intervals
Normal intervals (predicted value in parentheses)
Number of groups after students’
marking
Number of groups after moderation
weighting factor
Number of groups after moderation
weighting and adjustment factors
mean + σ (68,3%) 26(66,7%) 26(66,7%) 25(64,1%)
mean + 2σ (95,4%) 36(92,3%) 36(92,3%) 37(94,9%)
mean + 3σ (99,7%) 39(100%) 39(100%) 39(100%)
4.1.8 Spreadsheet Assignment 2 - Non-flow and Steady-flow Energy Problems and
90 round”, were made during the second assessment. Further reference to difficulties experienced with the computer exercises was mentioned during the interviews, discussed in section
4.4.3.1.1. These difficulties were not altogether unexpected as was mentioned earlier in Chapter 4.1.7.1 and Chapter 2.5.1.2, being first time peer assessors.
4.1.8.2 Analysis of the answers for Assignment 2
Of the assignment 2 assessment sheets that were handed in, only three answers were placed on the rubrics, none of which was correct. This could have been because of several reasons, such as:
the file was not available to assess or locked on the required assessment day, nine groups reporting that the file was not locatable
incorrectly calculated values were used to get the answer
incorrect input or output of the data or formulas, seven groups indicating that there were incorrect formulas or missing parts in the formula or formulas didn’t calculate anything
simply not writing the answers in the space provided
the exercise was too difficult.or too long
Considering the difficulty of the exercises, the following comments were made during the interviews, student H saying:
“Its more its its more of the programming, how to get your graphs right and it was...”, and student D said :
“...After assignment one, maybe we fin, we find it easier to do assignment two, because we knew what eww we did in ah for the first assignment, and we know we knew our problems were were about...”, and later
“...if I look at assignment two and then we looking back at us what we did in assignment one, what was our problems, we find it easier to do assignment two...”.
In a meta-analysis study comparing the effects of types of learning skills interventions by Hattie, Biggs and Purdie (1996) they rated various interventions and came up with an “overall effect size” according to the type of intervention. Computer-assisted instruction was rated at 0,31, where “the typical effect size in educational interventions was 0.40” (Hattie, Biggs &
Purdie, 1996, p.114). This indicates that computer-based instruction rates below the average.
91 However, part of this intervention included students’ peer-evaluation of other students’
assignments, which could be considered a type of remediation or reinforcement. In the Learning Strategies section of Hattie, Biggs and Purdie (1996), the Remediation/feedback was rated at 0,65, and Reinforcement, rated at 1,13, both above the average of 0,40.
When referring to the assessment of another groups work during the interviews, student D said:
“...when we see do what is right, then you put in, plug in the values, ya did come up with the answer, so the students who ah did that assignment, they knew what they were doing. And then we learnt from them to do a a assignment two...”.
Applying the same effect size calculation described in Hattie, Biggs and Purdie (1996, p.111), the Researcher came up with a figure of 0,13 for computer-assisted instruction which, although a positive figure, was considerably lower than 0,31 previously mentioned. One could interpret this as the style of intervention not being a significant contributor to the students learning in this study or that possibly the tasks were more difficult for the students than the Researcher
anticipated, although the latter one is ambiguous without further comparison to other similar studies, or feedback from the students themselves. The interviews did reveal that the computer intervention was initially seen to be difficult by some students, as detailed later in Chapter 4.4.3.1.1.
4.1.8.3 The moderation
Of the twenty-one files assessed by peers, seven were remarked by the moderators, representing a 30% moderation load, again more than the 10% specified on the assignment form. Too small a sample would have been generated by the moderators using only 10%. Similarly to assignment 1, the averages of the peers and the moderators’ averages were compared and a moderation weighting factor generated. This was 0,88821, a smaller figure than for assignment 1. Again, if the difference between the moderators’ marks and the peers’ marks was 25% or more, the average of the two marks became the final mark for the assessment. This occurred in only two instances, where one was under and one was over. The final analysis can be seen in Graph 4.3.
As the sample size was significantly less than the first assignment, the graph begins to show an inability to become normalised as seen by the final graph shapes of both the pre- and post- moderated curves. However, the moderated mark begins to show a more normalised Ogive shape.
92 GRAPH 4.3: Graph of comparison of Ogive before and after moderation for
Assignment 2.
A further statistical analysis of the marks using Quattro Pro version 9 was later undertaken to evaluate the moderation exercise. The results appear in Table 4.7(a) and (b) below.
Table 4.7(a): Statistical analysis of moderation of Assignment 2 Statistical data Students’
generated marks
Moderator’s mark
Marks after moderation weighting factor applied
Marks after moderation weighting factor
and adjustment factor applied
Readings, n 20 6 20 20
Average, µ(%) 65,3 62,0 58,0 57,6
Population standard deviation, σ(%)
17,0 15,1 14,8
Sample standard deviation, σ(%)
15,7
Interpreting the data of Table 4.7(b), one can draw similar conclusions to those for assignment 1. The moderation weighting factor did not change the distribution in any way, while the adjustment factor had a bigger impact on the distribution. Reference to Walpole, and Meyers (1978, p.513) standard normal population distribution, detailed in 4.1.4.5, again applies.
0 20 40 60 80 100
Cumulative Frequency (%)
0 2 4 6 8 10
Intervals
peer assessment before moderation peer assessment after moderation
Graph of comparison of Ogive before and after Moderation for Assignment
93 Table 4.7(b): Predicted and actual numbers of groups in normal intervals
Normal intervals
Number of groups after students
marking
Number of groups after moderation weighting
factor
Number of groups after moderation weighting and
adjustment factors
Mean +σ 14(70%) 14(70%) 13(65%)
Mean + 2σ 19(95%) 19(95%) 19(95%)
Mean + 3σ 20(100%) 20(100%) 20(100%)
4.1.8.4 Reassessment of assignment 2
During a lecture soon after the end of the whole intervention, there was a request by various members of the class to redo assessment two as several things had gone wrong during that session. These included various computer related issues, mainly with the networked system, virus problems and so on, as previously discussed in 4.1.4. The Researcher agreed and a date was fixed for later in the semester for any group who wished to redo the task. This falls in line with the SAQA guidelines on assessment (South African Qualifications Authority, 2001, pp.53- 54), where the same task can be performed again under the same conditions using the same instruments. Students were also allowed to work on the assignments in the meantime if they wished to improve their spreadsheets. Six groups went back later in the semester to redo the assessment of assignment 2. At the re-assessment each group had a rubric with a different problem from the original one, thus testing their spreadsheet on a fresh task, as recommended by the SAQA assessment guidelines (2001, p.54).
Again, each group member who signed the declaration on the assessment day declared that they would be allocated equal marks unless they specified otherwise. These re-assessment marks replaced the original ones and thus counted towards the final mark.
These new marks were added to the records overwriting their original marks, the same processes for moderation taking place in the same manner as previously discussed in Chapter 4.1.8.3.
94