CHAPTER 5 RESULTS SHOWING SCOPE AND QUALITY OF RESEARCH ON
5.2 SEARCHING AND SCREENING THE REPORTS
not in an educational setting; instead it focused on conceptions of nurses. Criterion 5 excluded a number of reports with unsuitable research aims; to be specific, five evaluated teaching programmes giving only quantitative data, another five were textbook analyses, 11 more gave outlines of aspects of acid-base models so they clarified propositional knowledge but did not include data on student conceptions, and a further 20 were teaching suggestions with no accompanying research. Seven reports duplicated data already published elsewhere; for instance Schmidt (1997) is a reinterpretation of data published earlier in Schmidt (1991; 1995) and Banerjee (1991) reported similar data to that in Schmidt (1995), but added nothing further to the research. A further two reports which gave unsuitable data (in that it was not in the form of quotations) were also eliminated through criterion 6. Finally, one report which was not available in the original was also included according to criterion 6. In summary, comprehensive searching identified 101 reports which were obtained and on scrutiny, 41 reports met all the necessary criteria, with data from one more having to be obtained from secondary sources.
These 42 reports then answer Research sub-question 1a: Which reports give suitable research data on student conceptual difficulties in acid-base chemistry? Detailed references for these 42 publications can be found in Appendix 1 (Page 272) as well as appearing as general references.
The body of work represented by the 42 reports had been published over a period of 28 years, with most of it published since 2000, confirming a recently increased awareness of student difficulties (see Section 2.4.1).
It is noteworthy that besides research studies on student conceptions; the initial search identified an appreciable amount (mostly from the Journal of Chemical Education) which clarified acid-base concepts (11 papers) and gave teaching suggestions for the topic (20 papers).
I found these publications a useful resource for propositional knowledge statements (see Sections 4.6.2). They also reflect a continued effort by various practitioners to lift the quality of instruction in this topic. However, despite this wealth of information, researchers have been sufficiently concerned about authors’ treatment of the topic to undertake six analyses of textbooks across the world. This suggests that authors of textbooks are not heeding this guidance along with the body of research into student conceptions (42 papers), just as Gabel (1999) indicated (see Section 2.4.4).
Perhaps textbook authors had not had access to suitable publications. Furthermore, in accordance with advice from Bennett et al. (2005a; 2005b) multiple search strategies had been used to identify suitable publications, which had been tedious and could perhaps be streamlined.
These concerns prompted analysis of ways to identify research reports. Results of search
strategies for the primary sources are given individually in Table 5.1. In addition Table 5.3 summarises data on the effectiveness of different strategies in identifying suitable primary sources of research. It can be seen from this data that only 19% of the reports were identified through more than one electronic source. Therefore a variety of strategies were indeed necessary in order to identify a wide range of reports.
Table 5.3 Effectiveness of various means of search strategies
Source of Reference *Number of reports found Percentage of 41 reports
In 2 or more electronic sources 6 15
ERIC 13 32
PsycINFO 1 2
Google Scholar 20 49
Academic Search Complete 3 7
Academic Search Complete alone 1 2
Contents alone 4 10
Cited alone 2 5
Personal contact alone 2 5
*Some reports appeared on several databases or search engines; consequently the numbers in the table do not total 41
Data in Table 5.3 also shows that the most productive electronic searches were through Google Scholar and ERIC with, respectively, 13 and 20 of the reports being on these databases, while PsycINFO was the least helpful, having identified only one item (and that one had already been identified through ERIC). Academic Search Complete database was fruitful for only 7% of the reports, although one item (Bradley & Mosimege, 1998) would have been missed without this database. A total of eight (or nearly one fifth) of the items needed more tedious strategies, specifically journal contents searches, following up citations or using personal contacts. This indicates that these more onerous methods should not be neglected. From this analysis of search results it appears that PsycINFO database added no further value to the search for research into student conceptions in acid-base chemistry. However, is clear that in order to conduct a comprehensive review, one cannot rely on only one database, nor can one rely only on electronic strategies. The variety of search strategies needed to identify research into student conceptual difficulties in acid-base chemistry confirms the recommendation by Bennett et al.
(2005a; 2005b) to use such diverse methods for a systematic review. Had this search been limited to databases, 20% of the reports would not have been identified. Additionally, apart from Google Scholar, these sources may not be accessible to teachers outside a university environment. This shows that searching a variety of academic databases remains a good
strategy to identify suitable research but these findings are not readily available to secondary or elementary school teachers.
The nature of the journals in which research was published was also analysed (see Table5.4).
This would add further insight into availability of research findings for teachers. In addition, the quality of the publications could give insight into the potential quality of the research published therein.
Table 5.4 Distribution of reports across journals
Journal Number of
reports
*International (or European) Journal of Science Education 9
*Journal of Research in Science Teaching 4
*@Chemistry Education: Research and Practice 3
*Research in Science Education 4
*@Journal of Chemical Education 3
*South African Journal of Chemistry 1
*@Biochemistry and Molecular Biology Education 1
#@School Science Review 5
#@Australian Journal of Education in Chemistry 1
#African Journal of Research in Science, Mathematics and Technology Education 1
#Journal of Baltic Science Education 1
#@The Chemical Educator 1
#Science & Education 1
Chemical Education International 1
Education Sciences: Theory and Practice 1
@Journal of Geoscience Education 1
Science Education International 1
Conferences: NARST 1998, NASTA 2004 2
*Appears on Science Citation Index of Institute of Scientific Information (ISI)
# Not on ISI index but editorial policy includes peer-review
@ Includes educational; practitioners in target audience
Data in the table shows that more than half of the reports come from journals which appear on the Science Citation Index of the Institute of Scientific Information (marked *) while a further quarter of the reports were from journals with a peer-review editorial policy published on their websites (marked #). This suggests that much of the research should meet international research quality standards; although these reports may have been published before a journal achieved such status. Furthermore 15 reports came from journals which include educational practitioners in their target audience (marked @). This distribution shows that researchers have not only
been publishing for their peers as Jenkins (2000) suggests. Instead, there has been some move towards making research outcomes available to practitioners as well.
In summary, in answer to research question 1a, extensive searches, using academic databases as well as non-electronic means, identified 101 reports which were subsequently obtained. After scrutiny, 41 reports (mostly published this century) were found to be suitable for critical analysis. They were published in journals with a target audience including both researchers and educational practitioners, with more than half of the reports appearing in internationally recognised peer reviewed journals. A secondary source, as cited by other authors, was also included.