CHAPTER THREE: RESEARCH DESIGN AND METHODOLOGY
3.5 Data Instrumentation and Collection
3.5.2 Quantitative methodology
3.5.2.1 Statistical data analysis of Learner Cognitive Data
157 In summary, the nature of qualitative research was appropriate for the study requirement of investigating the gifted programs offered in numerous schools, constructing multiple realities from the information collated from all educators, with generalizability of data not being the intent of this study (Lichtman 2010). Some of the findings were followed up by quantitative analysis of student data across their cognitive and affective domains and are discussed in the subsequent section of the research methodology.
158 an effective student profile of their basic capabilities, learner preferences, strengths, and weaknesses. The CAT4 suite of assessments comprise of the following constituents:
o Deduction with words (Verbal) o Working with numbers (Quantitative)
o Thinking with designs and shapes (Nonverbal)
o Reasoning with detailed and accurate shapes in 3-Dimensions (Spatial) The overall pattern of student thinking capabilities helps educators reach a better understanding of their unique learning needs and plan appropriately. The educational experts have accentuated the measurement of relational thinking or perception of the correlations between all the four aspects of cognition to understand the patterns of learning displayed by the students. The CAT4 tests can be as shown in figure 3.3 (GL Assessments, 2020).
159 Figure 3.3: Cognitive Ability Testing-Version 4 Batteries (GL Assessments 2020) While the verbal scores indicate the students’ language abilities and the quantitative data measure their mathematical proficiency, the nonverbal scores reveal basic reasoning with shapes and problem-solving skills thereby supporting in the understanding of their general cognitive ability. The spatial battery was a recent addition to evaluate how learners can manipulate precise objects and shapes while recalling this information in their memory. This critical skill is indicative of their inclination towards specific careers like engineering, photography, mathematics, astronomy, architecture, graphic designing, or physical sciences (GL Assessments, 2018).
160 In continuation, the nonverbal score is indicative of the students’ overall ability while their verbal or quantitative score may not be truly representative of their thinking. Any gaps in these scores may be helpful in identifying the possible reasons like specific learning difficulties, poor educational background, or English not being their first language of communication. As explained, the verbal data (inner voice) or the spatial processing (inner eye) combined with the nonverbal and quantitative scores correlating to how the student works with both the above extreme thinking skills supports the development of a comprehensive learner profile. The nonverbal and spatial scores are important predictors of academic attainment, help in identifying students with English as an additional language needs, influence of their cultural background, and their general ability as these are not dependent on their prior knowledge. (GL Assessments, 2018).
Schools can use the CAT4 data to identify gifted learners, make suitable differentiation in teaching and learning, track student progress, identify the individual barriers and promote improvement in student attainment with provision of appropriate challenges or interventions, as appropriate. Different levels of CAT4 assessments are administered according to the year groups of students. Student performance can be interpreted by the Standard Age Scores (SAS), or Stanines. (GL Assessments, 2018). The CAT4 data scores can be categorized as follows:
161 Table 3.5: The Stanine Scale (GL Assessments 2018)
As seen the scores above 119 in any CAT4 battery indicate High Ability and scores of 127 and above indicate Very High Ability. The CAT4 data can be used with other standardised assessments like the Progress tests to measure student attainment in relation to their ability (GL Assessments, 2018). The present study used the Stanine scales to correlate the participating student ability data to their individual attainment data.
In addition to the CAT4, it is mandatory for schools to conduct standardised attainment assessments, depending on the curriculum followed at the end of each academic year. To be more specific, the Progress Tests in English, Mathematics and Science (PTE/PTM/PTS) are conducted by UK and some IB curriculum schools, IBT tests by few
162 IB Curriculum schools, NWEA MAP by US-curriculum schools and ASSET by Indian curriculum schools (DSIB 2017).
Attainment Data used by UK/IB Curriculum Schools (PTE/PTM/PTS)
The participating UK Curriculum School and one IB Curriculum School used the GL Assessments- Progress Tests in English, Mathematics and Science (PTE/PTM/PTS) to measure the learner’s attainment in terms of their understanding, knowledge, and application across the core subject areas.
The Progress tests help to determine the level of the following skills:
o English: reading comprehension, spelling and punctuations, and grammar across age-appropriate non-fiction and fiction passages.
o Mathematics: important mathematical skills and aspects including mental math.
o Science: student understanding of curriculum content in physics, chemistry, and biology alongside application of scientific skills.
These series of tests are used for benchmarking in the UK and approved by the KHDA to be used in Dubai. The Progress Tests (PT) are appropriate for middle school students across the three core subjects as explained above. The PT data mapped against the CAT4 data indicated if the student was performing in line with their academic ability, exceeding or underachieving. School leaders used this analysis to inform the teaching and learning practices to ensure students’ academic progression (GL Assessments, 2018).
163 Figure 3.4: The Stanine Scale with descriptors (GL Assessments 2018)
Like the CAT4 scores, the researcher used the Progress Tests Stanine scale to understand the correlations between the scores and descriptors shown in figure 3.4 above to gauge if the student attainment matched, exceeded or was below the expected scores.
Attainment Data used by IB Curriculum School (ACER)
One of the participating IB curriculum school used The International Benchmark Tests (IBT) by ACER to compare student’s achievement scores over time and grades. These standardised assessments were utilized across the core subjects of English, Mathematics, and Science (ACER 2021).
The IBT Scores were available as Scaled scores and Achievement Bands. Since the achievement Bands were described in the range of 1-9 and matched the CAT4 Stanine scales, these were used to correlate the student attainment to their ability data. Like the process described above, if the student performed as expected, above expected or below expected, the attainment data were coded and used for further analysis.
An example of the student achievement bands in Mathematics and English with descriptors is included herewith for reference (ACER 2021).
164 Additional details about the ACER IBT Assessments are enclosed within the Appendices of this thesis.
Figure 3.5: ACER Mathematics Achievement Scale (ACER 2021, p. 12)
165 Figure 3.6: ACER English Achievement Scale (ACER 2021, p. 4)
166
Attainment Data used by Indian Curriculum School (ASSET)
Assessment of Scholastic Skills through Educational Testing (ASSET) is a skill-based standardised test used by the Indian curriculum schools that participated in this present study. This assessment can be used for students from grades 3 to 10 across the core subjects of English, Mathematics, and Science. Each student receives feedback regarding their strengths and weaknesses to support their learning in a personalised manner while helping the school benchmark the student performance alongside providing insights to teachers on areas that need to be intervened (ASSET, 2021).
Figure 3.7: ASSET Data Sample (Adapted from ASSET 2021)
One example of an anonymised individualised student report received by a participating school is shown above. For the purposes of this study, the ASSET scores were also available in Stanines 1 to 9 and hence was easy to be mapped against the CAT4 Stanines to measure if the student’s performance was as Expected, Above Expected or Below Expected. Additional details of ASSET assessments are enclosed as an Appendix.
Attainment Data used by US Curriculum School (MAP)
The National Assessment of Educational Progress (NAEP) used by the US Curriculum School is the ongoing and internationally representative quantifiable measure of student
167 achievement across numerous subjects over time. The NAEP data could be used to identify the highly able and gifted learners. It was mandatory for schools to administer the NAEP every two years for evaluating reading and mathematics in Grades 4 and 8.
Additionally, students could be assessed across various subject areas using the NAEP.
The test outcomes are transferred in scaled scores between 0 to 300 in Science and Mathematics while the scaled scores for reading are between 0 to 500. The qualitative descriptors correlating to these scaled scores are in three levels: namely, Basic, or Proficient or Advanced. As an example, 346/500 is the boundary when the descriptor changes from Proficient to Advanced in reading. Scaled scores for groups of learners including their demographic data were available to educators (NAEP, 2021). Since the MAP Scores were not available in bands or Stanines of 1-9 but as percentile scores, the conversion chart of percentile to stanine scores was used to convert the MAP percentiles to stanines. This ensured consistency in data mapping procedures for the current study.
The researcher used the MAP scores to gauge if the students in the US Curriculum School attained as expected, above or below expected scores. Additional information regarding the NAEP MAP Assessments and the Percentiles to Stanine charts are enclosed as Appendices. An example of MAP report is shown below for reference (NAEP, 2021).
168 Figure 3.8: MAP Summary Growth Sample (NAEP, 2021)
To summarize, if the individual student’s PTE/PTM/PTS or IBT or ASSET or MAP scores were in the range matching to their CAT4 scores, it was an indication that students were performing as expected (E). Students could be performing above their ability if their attainment scores are above their CAT4 indication (AE). When a student was performing as per the indicated ability or above, the educators may be satisfied with the teaching and learning practices. In contrast, if learner attainment scores are below their CAT4 scores (BE), an underachievement is indicated. The results were compared for each of their English, Mathematics, and Science subject areas.
The percentage of students attaining as per their Expected, Above Expected or Below Expected was indicated in the data analysis section.
The student ability data mapped to their attainment data supported the evaluation of the effectiveness of the gifted educational programs offered by each school in their cognitive domain. The next subsection explains similar analysis in the learners’ affective domain.
169 3.5.2.2 Statistical Analysis of learner Affective Data
Academic Motivation Scale (AMS)
Student motivation can be negatively impacted by a lack of challenge by the standard curriculum. The significance of the positive correlation between a challenging gifted program and learner motivation has been highlighted by numerous eminent scholars (Gubbels, Segers and Verhoeven, 2014). Understanding the affective domain of the students being offered special services would contribute truly meaningful insights into the holistic development of gifted educational provisions to educators (Greene & D’
Oliveira, 2009). The current study utilized a questionnaire-based Academic Motivation Scale (AMS) tool for the assessing the various motivations of middle school gifted students.
The Academic Motivation Scale (AMS) tool used was originally developed by Vallerand et al. (1992). This questionnaire tool comprised of 28 items distributed across 7 subscales which consists of 3 categories of intrinsic motivation, 3 categories of extrinsic motivation and 1 category of amotivation.
Students rate their perceptions using a 7-point Likert Scale where ‘1’ indicates Does not correspond at all, ‘2-3’ indicates Corresponds a little, ‘4’ indicates Corresponds moderately, ‘5-6’ correlates to Corresponds a lot, and ‘7’ means Corresponds exactly.
All the subscales are comprehensively detailed in figure 3.9.
170 Amotivation: lack of any motivation or engagement
Figure 3.9: Academic Motivation Scale Continuum (illustrated from Brophy 2010, p.155) Previous studies using the AMS tool have demonstrated good reliability and validity with reported alpha values in the range of 0.62-0.90 in the secondary learners (Utvær &
Haugan, 2016). The questionnaires were sent to all the participating students liaised by the relevant educators responsible for gifted education of each of the schools. After allowing for an appropriate time frame to gather responses, reminders were sent on multiple occasions. This study did not offer any monetary incentives to elicit student responses. However, the Covid-19 circumstances proved detrimental and only 26 student responses could be obtained for the purposes of the current study.