Chapter 5: Design and Methodology
5.5. Instrument design and Coding Process
The instrument has its design roots in the larger study. The instrument from the larger study was initially designed to code for various aspects which included content coverage, mathematical proficiencies, level of cognitive demand and teachers content knowledge, pedagogical knowledge and pedagogical content knowledge. Columns were provided in the mathematical proficiency section to code for the five strands, viz. conceptual understanding, procedural fluency, strategic competence, adaptive reasoning and productive disposition. This served as the core feature in the development of the instrument for this study. Initial changes were
Noor Ally Promotion of Mathematical Proficiency
informed by the theoretical framework chapter. Sub-categories for each strand were introduced as understanding of mathematical proficiency increased resulting in changes in the instrument design. Ongoing subtle changes to the instrument occurred as the coding progressed and the theory of opportunity to develop mathematical proficiency (OTDMP) was formulated. This eventually culminated in an instrument which captured the essence of the study yet did not result in re-coding of the initial videos. The fundamental structure of the instrument remained intact which fore-grounded the theoretical and analytical framework. Changes included a section for recording observations in each segment. Here, the notes were descriptive and illustrated the material relevant to the research. These notes were later utilised to identify types of classroom interactions, for instance (see Chapter 6).
Factual descriptions were used to describe the observations, steering clear of interpretation or inferences. Example of recordings would take the following tone:
‘the teacher discusses an example of addition of mixed numbers, by initially converting to improper fractions.’ Inferential remarks such as, ‘he is trying to...’ or
‘an unsuitable diagram is chosen to....’ were avoided. These descriptions allowed the coder to return directly to a specific segment of the lesson if required to do so. A description of the opportunities which were identified was recorded in the observation part of the instrument.
A segment of the lesson was defined as the time interval within which mathematical instruction occurred. A segment of 5-minute duration was the unit of analysis in this study. Thus each lesson was viewed in intervals of 5-minutes in line with the larger project. A maximum of 12 segments was inserted to accommodate for maximum lesson duration of 60 minutes. Initially a stop – pause – analyse and record procedure was implemented. As the process was repeated and the familiarity of the descriptor table increased, the procedure changed into viewing and recording without pausing save for exceptional circumstances. This was aided in part by the fact that the software used to view the videos clearly showed the time lapsed at any point in the viewing cycle.
As alluded to earlier in this chapter, the instrument used in the larger study was adapted to mirror the requirements of this project. Additional columns and split
columns were inserted for each strand to record the finer grain size required for each opportunity. Columns were needed for OTDCU, OTDPF, OTDAR, OTDSC and OTDPD. Sub-categories were then used to provide further meaningful analysis for each strand of opportunity as discussed in the theoretical framework chapter.
Each lesson was then viewed in intervals of 5 minutes with each segment been assessed for the five strands of OTDMP. This coding instrument and rating was adapted from the larger study. This was necessary to analyse lessons using a finer grain size as opposed to that used in the larger study. A 2-step procedure of identification and rating was used.
Step 1: Analyse each interval and detect whether the opportunities were ‘present’ or
‘not present’.
The following lettering indicated this step: P – present NP- not present If ‘NP’, a ‘√’ was then inserted in the appropriate cell of the video coding sheet.
Step 2: If ‘present’ then the grading system using the descriptor table explained in the analytical framework chapter was implemented. The values 1, 2 or 3 were inserted when the observed mathematical opportunity correlated with the ratings in the descriptor table. The list below indicates how each opportunity appears in the final instrument that was used.
OTDCU: S – stated D – developed OTDCU
NP P
Stated Developed Figure 3: Instrument design for OTDCU
Noor Ally Promotion of Mathematical Proficiency
OTDPF: T – task controlled S – solver controlled OTDPF
NP P
Task Solver Figure 4: Instrument design for OTDPF
OTDSC: Pictures, lists, flowcharts, trial and error, patterns, like problems and others.
OTDSC N
P
P Pictures Lists Flowchart Trial and
error
Patterns Like problems
Other
Figure 5: Instrument design for OTDSC
OTDAR: Reasoning, justification and explanation.
OTDAR
NP P
Reasoning Explanation Justification Figure 6: Instrument design for OTDAR
OTDPD: Perseverance, confidence and real world.
OTDPD
NP P
Perseverance Confidence Real World Figure 7: Instrument design for OTDPD
An additional feature of the instrument included rows and columns used to collate the ratings for the respective strands. The insertion of rows was necessary during the analysis phase of the research. Results were collated by inserting formulae for the rows and columns that would allow comparative analysis. The aggregation of results within and across strands was used to answer the research questions.
The formulation of the notion of OTDMP followed by the descriptor table per opportunity incorporating the characteristics that inform the ratings contained sufficient information to assess the strands of mathematical proficiency for the lesson unit. Using the results of OTDMP, it would be possible to correlate the analysis of the videos with the results from the learner and teacher questionnaires and tests as well as with other variables. The notion of OTDMP will be the main focus and feature prominently during results and analysis. Appendix 2 shows the instrument used in the study.