CHAPTER 5 RESULTS SHOWING SCOPE AND QUALITY OF RESEARCH ON
5.4 QUALITY OF THE RESEARCH
5.4.1 High Quality Research
“students preferred one of the non-conjugate acid-base pairs, namely NH4
+/OH-” (p734). He then explored conceptions of high school students using open-ended questions related to relevant chemical equations for three reactions. From this “it seemed that the students did not merely confuse the terms conjugate and non-conjugate. They attempted to find a matched pair of ions; one with a single positive charge, and the other with a single negative charge” (p735).
Following this preliminary work two multiple-choice items, also requiring explanations, were prepared to investigate which pair of ions students would select. This instrument was administered to 160 from a selection of 4291 senior high school students and four group interviews were undertaken. The final descriptions are reported as: “they confuse non- conjugate and conjugate acid-base pairs” and “They regard positively and negatively charged ions as conjugate acid-base pairs” (p739).
The research sequences used by Schmidt, show how the description of the student conception changed: initially relating to a specific reaction then being phrased as a general student heuristic which more accurately represents student thinking. Schmidt also established neutral ground driven by students’ frame of reference (Johnson & Gott, 1996) in three ways. Firstly, the Brønsted model was known to be part of the curriculum for these students, secondly students used terminology related to Brønsted model even if they arrived at the wrong conclusion, and finally students from the same class could give plausible comments using the model. In these reports, Schmidt also shows how he interpreted data triangulated from different sources in different educational and chemical contexts against a description of acceptable chemistry to arrive at stable descriptions of student difficulties. Accordingly the research met criteria for Level 4 difficulty descriptions. It is therefore astonishing that at the time of searching (May 2006 to January 2008) Schmidt (1995) did not appear on the ERIC database (see Table 5.1).
Several other projects report rigorous research but for various reasons were of limited use in the present critical analysis. In the first of these, Chiu (2005; 2007) describes an extensive project of surveying student conceptions in many chemistry topics among different age groups in Taiwan. She does not give specific details for acid-base probes but does, however, carefully document, with examples, the process by which two-tier multiple-choice items (the first choice and then the explanation for the choice) probes were designed in other topics according to Treagust’s (1988; 1995) procedure. This involved preliminary open-ended written items and interviews, then piloting and validating the instrument. Although the acceptable propositional knowledge is not given explicitly, the procedure by which this had been validated through expert opinion and concept maps is also reported. The reports focus on an overall survey for the
country so much of the fine-textured qualitative data had been eliminated, thereby limiting its usefulness in this review and synthesis.
Some studies showed evidence of high quality research but with limited contexts. Further research could build on these to verify the stability of the description of a student difficulty across a greater range of contexts and so enable it to be termed Established. I next discuss two early research studies which show triangulated research that was limited to only one educational context, in these cases, single cohort of students. There is a problem with the manner in which such research results are subsequently cited.
The first project (Ross & Munby, 1991) started with a multiple-choice instrument administered to a single high school class. Items had been shown to be reliable and valid from a pilot study.
This was followed by two rounds of interviews that were conducted with students selected from the initial group. Data was then triangulated from the three sources to give a more complete
‘picture’ for conceptions among the single student cohort. In the second project, (Nakhleh &
Krajcik, 1993; 1994; Nakhleh, 1994) collected data through two sets of interviews, pre- and post-instruction. These were combined with personal observation of the class involving protocol analysis of student verbal commentaries and discussion that occurred during laboratory exercises. The interview sequence is described, it had previously been piloted, and four experts had validated the content. Both projects therefore used several means of data collection in accordance with principles of triangulation. Interviews in the latter project were conducted on neutral ground. This aspect was shown by the interviewer asking the student: “you mentioned ... could neutralize, what does neutralize mean to you?” (Nakhleh & Krajcik, 1994, p 1080).
Therefore this project also included data interpretation within students’ frame of reference.
Both sets of authors describe corresponding propositional knowledge, either as a concept map (Ross & Munby, 1991) or an explicit list (Nakhleh & Krajcik, 1994) and show how they interpreted student quotations against this. From one high school class, Nakhleh and Krajcik (1994) report five clusters of conceptions showing student difficulties. Ross and Munby’s focus is more on the method of author generated concept maps. Consequently, there is no thrust to describe frequently occurring conceptions; instead, they describe the conceptions of two students in detail with general reference to the conceptions found in rest of the class. These two studies show valid and reliable probes, supported by interview quotations interpreted against scientifically acceptable knowledge which are merged to give descriptions of student conceptions. Moreover, neither pair of authors makes claims about these results being applicable beyond the study cohorts.
The problem does not lie in these research projects, but the uncritical way in which other authors have subsequently cited their results, with no reservation that they came from a single cohort, or even single students. For example, Pinarbasi (2007, p 24) writes “Nakhleh and Krajcik (1994) established that ...” with similar statements in Lin and Chiu (2007). Likewise, Dhindsa (2002, p 21) writes about Ross and Munby’s work as “It has been known that students ...” Moreover, another problem arises when instead of taking this work, and building on it to be able to describe a conception more accurately (as did Schmidt) some authors use these findings without further investigation as distractors in their own multiple-choice probes. In particular, Demicioğlu et al. (2004; 2005) use “All acids have bubbles” (Nakhleh & Krajcik, 1994) without reporting further research. Consequently their data adds little to clarify the nature of a conception except that other students also chose these words. Other conceptions have been reported more recently through similarly triangulated quality research among single student cohorts (e.g. Demerouti et al., 2004; Watters & Watters, 2006; Sheppard, 2006; Furió-Más et al., 2007) and it remains to be seen how these will be subsequently cited. It appears that many later authors treat all reported conceptions as “established” needing no further investigation into their nature.