As described in Chapter 3, the researcher first conducted a pilot study together with four other raters by analysing the March 2013 paper one. After discussions and an
agreement on how to conduct the analysis was reached, the raters then analysed the
November 2012 paper one (physics) and thereafter analysed the November 2012 paper two (chemistry). The inter-rater reliability calculated in section 4.1.2 showed that the TASSK typology was a reliable instrument and the researcher used this instrument to analyse all the final NSC physical sciences examination papers set by the DBE.
Table 4.3 which follows shows the results of analysis of the physics examination question papers from 2008 to 2013. The first row indicates the percentage cognitive weighting that is recommended by the DBE. The DBE’s “application” is matched with the combined “routine execution” and “application” categories of the TASSK. This
comparison was discussed in section 3.5.5. The table below also shows the percentage per cognitive category for each year’s question paper. The mean percentage per cognitive category was calculated for the period 2008 to 2013 and the variation between the DBE recommendation and the mean percentage is indicated on the last row of the table.
On average over the six year period, 93% of the total marks in question paper one (Physics) were allocated to questions requiring lower order thinking skills of “retrieval”,
“basic comprehension” and “routine execution”. Only 7% of total marks were allocated to questions requiring higher order thinking skills of the “application” type. There were no marks allocated to test the higher order thinking skills of “analysis and reasoning” as described in the TASSK, the Department of Education (2008a) and the Department of Basic Education (2011a) taxonomies in any of the physics papers over the six year period from 2008 to 2013.
Table 4.3
Percentage of the total marks of Physical Sciences examination paper 1 (Physics) from 2008 to 2013 per cognitive level
Cognitive Domain
Lower order thinking skills Higher order thinking skills Retrieval
Level 1
Basic Comprehension
Level 2
Routine Execution
Level 3
Application Level 4
Analysis
& Reasoning Level 5
DBE % 15 30 45 10
2008 19 28 39 14 0
2009 17 31 43 9 0
2010 16 24 57 3 0
2011 18 24 52 6 0
2012 15 25 56 4 0
2013 19 26 48 7 0
Mean % 17 27 49 7 0
Variation % +2 -3 +11 -10
From table 4.3 it can be seen how the cognitive demand compares with the recommendations in the NCS subject assessment guidelines (Department of Education, 2008a). The “retrieval” category was over-emphasised in all question papers except the November 2012 paper. The “basic comprehension” was under-emphasised in all but the November 2009 question papers. The combined “application” to routine and new
situations was over-emphasised in all the question papers while the higher order thinking questions that involve problem solving, decision making, experimenting and investigating according to the TASSK typology did not feature at all during the study period.
Table 4.4 which follows shows the analysis results of the chemistry examination question papers from 2008 to 2013. The first row gives the DBE percentage cognitive weighting recommendations for chemistry which differs from that of physics. This is followed by the percentage per cognitive category of the TASSK typology for each year’s paper. The mean per cognitive category is worked out for the period 2008 to 2013 and the variation between the DBE recommendation and the mean percentage is indicated on the last row of the table.
Table 4.4
Percentage of the total marks of Physical Sciences examination paper 2 (Chemistry) from 2008 to 2013 per cognitive level
Cognitive Domain
Lower order thinking Higher order thinking Retrieval
Level 1
Basic Comprehension
Level 2
Routine Execution
Level 3
Application Level 4
Analysis
& Reasoning Level 5
DBE 15 40 35 10
2008 30 41 29 0 0
2009 18 47 28 7 0
2010 30 29 32 9 0
2011 25 43 32 0 0
2012 15 25 56 4 0
2013 18 28 50 4 0
Mean % 22 36 38 4 0
Variation % +7 -4 +7 -10
On average over the six year period, the findings of this study regarding paper two (Chemistry) reveal that 96% of the total marks were allocated to questions that tested skills requiring lower order thinking, while only 4% of the total marks were allocated to
questions that tested skills requiring higher order thinking skills of the “application” type.
The 2008 and 2011 chemistry question papers, however, did not include any of the
“application” type questions. There were no marks allocated to test the higher order thinking skills of “analysis and reasoning” as described in the TASSK, the Department of Education (2008a) and the Department of Basic Education (2011a) taxonomies in any of the chemistry papers over the six year period from 2008 to 2013. In comparison with the physics papers, the chemistry papers for this period appear to have a lower cognitive demand. This, however, must be seen in the context of the DBE recommended weighting of cognitive level 3 which constitute 45 % for physics and 35 % for chemistry.
From table 4.4 it can be seen how the cognitive demand compares with the recommendations in the NCS subject assessment guidelines (Department of Education, 2008a). The “retrieval” category was over-emphasised in all question papers except the November 2012 paper. The 2008 and 2010 question papers doubled the recommendations for the “retrieval” category indicating an emphasis on lower order memory skills. The
“basic comprehension” was under-emphasised in the November 2010, 2012 and 2013
chemistry question papers and over-emphasised in the 2009 and 2011 chemistry question papers. The combined “application” to routine and new situations was under-emphasised in the 2008 question paper and over-emphasised in all the other papers while the higher order thinking questions that involve problem solving, decision making, experimenting and investigating did not feature at all during the study period.
To answer the research question,
What are the cognitive demands of the NSC physical sciences examination question papers?
I combined the cognitive demand analysis of paper 1 and paper 2 since the DBE aggregates the NSC Physical sciences examination paper 1 (physics) and paper 2
(chemistry) and arrives at examination mark for the learner. The cognitive demand levels for the NSC physical sciences examination is reflected in table 4.5 below.
Table 4.5
Combined percentages of marks per cognitive level for paper 1 (Physics) and paper 2 (Chemistry) Lower order thinking skills Higher order thinking skills Cognitive Domain Retrieval
Level 1
Basic Comprehension
Routine Execution
Application Level 4
Analysis &
Reasoning
Level 2 Level 3 Level 5
DBE Mean % 15 35 40 10
Mean P1 (Physics) 17 27 49 7 0
Mean P2 (Chemistry) 22 36 38 4 0
Mean % (P1 & P2) Variation %
19 +4
32 -3
43 6
+9
0 -10 On average over the six year period, the findings of this study reveal 94% of the total marks were allocated to test lower order cognitive demand in the physical sciences NSC papers from 2008 to 2013 according to the TASSK cognitive demand typology.
Application skills accounted for the remaining 6% of the total mark allocation, while 0%
of total marks accommodated the testing of higher order thinking questions involving
“analysis and reasoning” using this taxonomy.