4.8 Instrumentation
4.8.1 The Learners’ Functional Understanding of Proof (LFUP) scale
Shongwe and Mudaly (2017) undertook a methodological study whose purpose was twofold: to develop an objective instrument to measure Grade 11 learners’ functional understanding of proof in mathematics. At the time of its development, the instrument was referred to as the Functional Understandings of Proof Scale (FUPS). They conducted an exploratory study in two stages: (1) theoretical development of subscales and items and (2) field-tested the instrument and determined its psychometric properties by randomly surveying two groups of participants: 37 mathematics participants and 37 mathematical literacy participants. Mathematical literacy is an FET phase subject that applies mathematical concepts to everyday situations; for example, calculating income tax transfer fees, legal fees, and bond repayment, reading and interpreting statistics in newspaper articles (Clark, 2012).
Research methodology Instrumentation
Initially, a 31-item questionnaire was developed. A panel of experts evaluated content validity and the known-groups method was adopted to assess construct validity. For reliability, internal consistency and item-total correlations were assessed. The instrument received an overall reliability coefficient of .886. In the final analysis, the scale consisted of 25 Likert items. Having given a background to the previous version of the measurement instrument, I now describe the new version (LFUP). In this study it is referred to as learners’ functional understanding of proof (LFUP) scale.
4.8.1.2 The LFUP questionnaire for the present study
The validation of the LFUP scale in this study was an effort to provide teachers and educators with an instrument to measure learners’ functional understanding of proof and thus inform classroom practice. A curriculum geared towards reflecting the mathematics discipline needs to incorporate the functions of proof in mathematics. As already mentioned, the FET phase mathematics CAPS curriculum stipulated Specific Aims, one of which is understanding that the learning of proof without grasping why it is important, leaves learners ill-equipped to use their knowledge later in their lives. However, effective endeavours aimed at developing learners’ informed views of the functions of mathematics require a clearer picture of the current baseline views of these functions:
verification, explanation, communication, discovery, and systematisation.
Efforts to evaluate the LFUP scale in this study were guided by the evidence-centred assessment design (ECD) framework. This design framework is based on the principles of evidentiary reasoning embedded in advances in cognitive psychology on how learners gain and use knowledge (Mislevy, Almond, & Lukas, 2003). As Mislevy et al. (2003) put it, ‘designing assessment products in such a framework ensures that the way in which evidence is gathered and interpreted is consistent with the underlying knowledge and purposes the assessment is intended to address’ (p. 2). This is important in order to provide teachers and teacher educators with information from which accurate instructional decisions can be taken.
Research methodology Instrumentation
In conjunction with Kane’s (2004) work, the Standards for Educational and Psychological Testing15 was the basis on which the validity and reliability of the LFUP instrument were framed.
The standards are intended to promote sound and ethical use of tests and to provide a basis for evaluating the quality of testing practices. Hill, Ball, and Schilling (2008) have indeed found the standards appropriate and express their belief that for any measurement development effort, data obtained from pilot testing of study items must be analysed to assess whether the instrument meets several measurement-related criteria for it to yield trustworthy results.
Following a trawl of the literature around the concept of proof functions, the structure of LFUP scale has also been modelled on those that were used by Almedia (2000), Ruthven and Coe (1994), and Schoenfeld (1989). These instruments consisted of items that participants typically check-marked on Likert scales ranging from “very true” to “not at all true” and from “strongly agree” to “strongly disagree”. The questionnaires contained items such as, “Proof is essential in pure mathematics” or “The key thing is to get the statements and reasons in proper form”.
However, some aspects of these questionnaires were found unsuitable for this study for two main reasons. The first is that, unlike in this study, the exploration of proof was not limited to Euclidean proof only. The second is that in this study the key focus area was on exploring functional understanding of Euclidean proof rather than exploring the value of proof in other areas of mathematics. Therefore, the questionnaires were not entirely aligned with the objective of this study.
In this study, quantitative data was collected through administration of a five-point Likert scale questionnaire (LFUP scale in Appendix B1) for analysis in order to answer research question,
“What functional understanding of proof do Grade 11 learners hold?” The first section of the LFUP questionnaire contains items for gathering demographic data: gender, class name, and home language (Table 4—1). Taking into account Kumar’s (2005) guidelines for formulating questions,
15 For a detailed discussion of these standards, the reader is directed to the manual published jointly by the American Educational Research Association, the American Psychological Association, and the National Council on
Measurement in Education (2014).
Research methodology Instrumentation
every effort was made to ensure that simple and everyday language in the questionnaire was used for two reasons. First, English was not the home language of most of the participants. Second, there was no time allocated for explaining the questions to the participants. The purpose of collecting demographics was to be able to adequately describe the sample. Ensuring that language used was appropriate because misunderstanding of the questions by participants would have resulted in irrelevant responses.
Table 4—1. The structure of LFUP questionnaire
Category Description Number of
items
Demographics Code; Gender; Class; Home Language 4
Verification function Five-point Likert scale assessing
understanding of proof as a means to verify
3 Explanation function Five-point Likert scale assessing
understanding of proof as a means to explain
5 Communication function Five-point Likert scale assessing
understanding of proof as a means to communicate
5
Discovery function Five-point Likert scale assessing understanding of proof as a means to discover/invent
5
Systematisation function Five-point Likert scale assessing understanding of proof as a means to systematise
7
The second section of the LFUP questionnaire has 25 Likert scale items that range from 1 (“Strongly disagree”) to 5 (“Strongly agree”). The scores on the LFUP scale were treated as interval level scale which was amenable to parametric statistical analyses. There are five dimensions (factors) in the LFUP questionnaire, organised as follows: (1) verification; (2) explanation; (3) communication; (4) discovery; and (5) systematisation. A sample of the explanation function and its associated items is shown in Table 4—2.
Research methodology Instrumentation
Table 4—2. An extract showing items of the Explanation scale on the LFUP instrument (n = 135)
Item SD D N A SA
T4 A proof explains what a maths proposition means. 1 2 3 4 5 T5 A proof hides how a conclusion that a certain maths
proposition is true is reached.
1 2 3 4 5
T6 Proof shows that maths is made of connected concepts and procedures.
1 2 3 4 5
T7 When I do a proof, I get a better understanding of mathematical thinking.
1 2 3 4 5
T8 Proving make me understand how I proceeded from the given propositions to the conclusion.
1 2 3 4 5
SD = strongly disagree; D = disagree; N = neutral; A = agree; SA = strongly agree