Paper 1: Vocabulary Acquisition in School-age Children: Development and Application of the Kilifi Naming Test
Introduction
Few studies report the measurement of expressive vocabulary in school-age children in resource-constrained settings. Furthermore, only a small number of locally developed standardised norm-referenced measures of language functions have been published for use with the multiple language groups of sub-Saharan Africa (SSA). This makes it difficult to detect language and communication problems especially among school-age children who may be wrongly diagnosed as having a general learning disability. Our current understanding of influences on vocabulary acquisition is generally limited to those linguistic and cultural contexts where standardised tests of vocabulary are available.
Vocabulary acquisition is incremental in nature (Laufer & Goldstein, 2004) and words that are encountered more frequently are learned more quickly (Huttenlocher, Haight, Bryk, Seltzer, & Lyons, 1991). For a child, the progressive and continual acquisition of new vocabulary is important for the development of communication skills (Smith, Cowie, & Blades, 2003). Assessing vocabulary enables us to identify children with potential literacy problems such as reading comprehension as the two are highly related (Sénéchal, Ouellette, & Rodney, 2006).
The multi-directional interactions between biological (internal) factors and environmental (external) inputs, couched within Bronfenbrenner’s bioecological model (Bronfenbrenner, 1995), have a strong influence on children’s vocabulary acquisition (Hart & Risley, 1995; Hoff, 2003) even among socioeconomically disadvantaged populations (Apiwattanalunggarn &
Luster, 2005; Hamadani et al., 2010; Weizman & Snow, 2001). Internal (i.e. child attributes) and external (i.e. features of the home environment) factors, including gender, age, nutritional status, school exposure, availability of household resources and neighbourhood of residence may underlie the substantial variability observed in vocabulary acquisition among children. An examination of these factors will shed light on the potential causes of differences in vocabulary acquisition.
Several study findings attest to the fact that children show vast improvements in vocabulary acquisition with increasing age (Basilio, Puccini, Silva, & Pedromónico, 2005;
Bates, Dale, & Thal, 1995). Considerable variations are thus recorded across children, even those of the same age.
The research has not clearly established if gender differences occur at certain ages or may be attributed to innate biological differences or external environmental and social factors (Bornstein, Hahn, & Haynes, 2004; Burman, Bitan, & Booth, 2008; Leaper, 2002; Maccoby,
36
1980). However, the reported differences between boys and girls tend to be small, inconsistent (Hyde & Linn, 1988) and not always significant.
With regard to poor nutritional status, its negative effects include a shortened attention span, reduced capacity (Sigman, Neumann, Carter, et al., 1989) and little energy to learn (Brown & Pollitt, 1996) resulting in lower scores on various outcomes, including vocabulary tests. Unfortunately, these effects usually begin early in a child’s life and the resultant disadvantages are long-lasting (Grantham-McGregor et al., 2007; Mendez & Adair, 1999).
Poor nutritional status affects a significant proportion of children living in resource-poor settings and because of its persistent effects, it becomes very important to prevent it early on in a child’s life.
Larger socioeconomic structures such as the neighbourhoods in which children live influence children’s outcomes indirectly through various proximal social contexts such as families and schools (Bronfenbrenner, 1979; Leventhal & Brooks-Gunn, 2000; Sampson, Morenoff, & Gannon-Rowley, 2002). This association varies by the extent of neighbourhood advantage (Dupéré, Leventhal, Crosnoe, & Dion, 2010) so children living in neighbourhoods with more resources are likely to have better outcomes.
At the family level, socioeconomic status (SES) affects the manner in which adults use language with their children. The number of different words and the total number of words spoken during these interactions will determine the size of a child’s vocabulary (Hart & Risley, 1995; Weizman & Snow, 2001). Parents with more socioeconomic resources at their disposal more frequently talk with the aim of eliciting conversation, use longer sentences and a richer vocabulary than those with less (Hoff-Ginsberg, 1991; Hoff, 2003). On the other hand, poorly educated parents living in crowded homes are less verbally responsive to their children, use less diverse language and their speech more frequently serves the function of directing the child’s behaviour (Evans, Maxwell, & Hart, 1999; Hoff, Laursen, & Tardif, 2002). Not surprisingly, poorer outcomes have been reported for children living in homes with fewer resources at their disposal (Hart & Risley, 1995) as children in homes where they are exposed to fewer words develop smaller vocabularies. Substantial differences in vocabulary size among school-age children are reported, with estimates ranging from 6,000 to 14,000 words (Weizman & Snow, 2001). The growing body of evidence linking early vocabulary development and subsequent successful acquisition of reading and writing skills (Sylva, Melhuish, Sammons, & Siraj- Blatchford, 2000) underlies the importance of meaningful interactions between parents and their children.
By the time they get to school, most of the words that children encounter in their everyday conversations are already in their vocabulary repertoires (Cunningham & Stanovich, 1998; Hayes & Ahrens, 1988). Children may however pick up new words through incidental exposure; for example, during their play sessions in school, they may hear new words whose
37
use in context enable them to deduce their meanings (Connor, Morrison, & Slominski, 2006;
Miller & Gildea, 1987) without any explicit training or feedback (Bloom, 2000). Although some studies in Western settings suggest that additional years in school do not have a measurable impact on vocabulary growth in children especially during the early school years (Cantalini, 1987; Chall, Jacobs, & Baldwin, 1990; Christian, Morrison, Frazier, & Massetti, 2000; Skibbe, Connor, Morrison, & Jewkes, 2011), other studies have shown that since language is a socially-mediated process, teachers provide children with opportunities for vocabulary learning through their daily oral language discourse (Huttenlocher, Vasilyeva, Cymerman, & Levine, 2002). Teacher input may therefore have a strong impact on vocabulary growth in children as teachers provide high quality language modelling at school (Penno, Wilkinson, & Moore, 2002). This may be especially true in resource-constricted settings where the provision of education is not universal, and children not in school may not benefit from a similar influence from their parents at home.
To assess expressive vocabulary, we chose to use confrontation naming which is sensitive to brain injury (R. W. Cheung, Cheung, & Chan, 2004; Jordan & Ashton, 1996). Furthermore, the measures used tap cognitive skills such as encoding and retrieval. Expressive vocabulary tests show strong relationships with other aspects of oral language and therefore more accurately reflect emergent literacy (Malvern & Richards, 1997). Such measures can thus serve as proxies for reading comprehension specifically and academic achievement more generally (R. C. Anderson & Freebody, 1981; Beck, McKeown, & Kucan, 2002; Cunningham &
Stanovich, 1998; Dickinson & Tabors, 2001).
We considered confrontation naming a suitable approach because compared to younger children, most school-age children possess naming abilities and are able to verbalise their responses. Whereas receptive vocabulary tests do not require reading, writing or speaking during assessment, they are more costly and complex to produce and require more time to administer than expressive vocabulary tests. This, coupled with the problem of providing sufficient drawings recognisable to the children in the current study, provided the impetus for developing a confrontation naming test for this age group. Also, the requirement to choose from a selection of available items bears little relation to the way language is used in most real- life situations (Luo & Zhang, 2011). This may make the test format more susceptible to guessing and impulsive responding than tests requiring an open-answer format (Luo & Zhang, 2011). A similar response bias where children picked a picture from the same position was observed in the administration of a picture vocabulary test of comprehension designed for children aged 5 to 9 years in the same setting as the current study (Holding et al., 2004).
Furthermore, at this age children have appropriate levels of comprehension and concentration making such a method more sensitive (Clacherty & Kushlik, 2004). This procedure therefore provides a more direct assessment of vocabulary skills than would be obtained using parental
38 reports or observation of communicative interactions.
There have been studies on language development in children in the sub-Saharan African setting; however, they are few in number, have mostly utilised small sample sizes (Carter et al., 2006; Carter et al., 2005) or have relied on ‘Western’ instruments to measure child outcome (Msellati et al., 1993; van Rie, Mupuala, & Dow, 2008). In relation to the latter point, Alcock and colleagues (2007) have noted that the problem with using instruments not specifically developed for the population under study may result in data that suggest language delay or demonstrate lack of sensitivity to individual differences in the population of interest. Whilst some earlier studies have compared the rates of development of speech among different language groups (Demuth, 1990; Suzman, 1987), others have focussed on the influence of illness, nutritional supplementation and various environmental factors on various aspects of language functioning in children. And although other studies have reported the adaptation and use of measures of comprehension vocabulary among infants, pre-school and school-age children living in resource-constricted settings (Bortz, 1995; Holding et al., 2004; Pakendorf &
Alant, 1997; Sigman, Neumann, Carter, et al., 1989), some of the measures were limited by their lack of sensitivity to brain injury at younger ages. As far as the literature search has revealed, we are not aware of any efforts to create a standardised assessment of expressive vocabulary for school-age populations in SSA.
In designing a vocabulary measure for rural school-aged children, context-specific cultural and language differences present translation difficulties. Hence it would not be valid to apply any of the available published measures, such as the Boston Naming Test (BNT; Kaplan, Goodglass, & Weintraub, 1983). This study is unique in that assessment of expressive vocabulary among a school-age population has not been previously conducted in this setting.
Considering how varied and complex language is, our study did not however seek to distinguish language delays and disorders; rather, we were more interested in describing variability in vocabulary acquisition as an important element of global cognitive functioning. Our primary purpose was to identify a list of words that would be useful in creating a measure of vocabulary development in a rural community of school-age children. Secondary aims were to establish the psychometric properties of the measure, examine sources of variability in vocabulary acquisition, and investigate associations between children’s vocabulary scores and performance on measures of non-verbal reasoning and educational achievement.
Method
The details of the study setting and sample are presented in Chapter 3.
Study Setting
The study was conducted in Kilifi District, Kenya.
Study Sample
The sample consisted of 308 boys and girls.
39 Procedures
Test design. In designing the test, we had a number of objectives – that the test would be: simple and quick to administer; require no specialised equipment; and, elicit clear, responses that are easy to record. We therefore developed a test similar to the BNT in terms of structure, administration and scoring that would be appropriate for school-age children (eight years and above). Different versions of the BNT have been widely used to investigate naming or word retrieval (Kim & Na, 2007; Miotto, Sato, Lucia, Camargo, & Scaff, 2010; Storms, Saerens, &
Deyn, 2004). The BNT also provided an appropriate framework for length, and was used to suggest possible categories of words.
The KNT was developed following the 4-step systematic test adaptation procedure outlined by Holding and colleagues (2009). We first defined the construct after which we created the item pool. We then developed the procedure for administering the test and four assessors were trained in administration and scoring. In the fourth and final step, we evaluated the adapted schedule to ensure that the test was appropriate for the age for which it was intended. The details of this process are presented in Chapter 3.
In the KNT, the child is asked to spontaneously give one-word responses when presented with a black and white line drawing of a familiar object. All children were tested individually in a quiet room. Participants were administered the items in a standard order beginning with item 1. No time limits were imposed for responding. If a child provided the correct response, i.e. the name of the item written on the record sheet, the assessor recorded ‘C’ on the record sheet. A stimulus cue was provided when no response was given, the child stated that s/he did not know the name or the item was incorrectly perceived (e.g., if the child misperceived a saucepan as a cup, s/he would be given the cue that the item was used “for cooking”). If provision of a stimulus cue did not result in a correct answer, the word that the child provided was recorded verbatim as a non-target word response. If a child failed to correctly name any objects on six consecutive trials, the test was discontinued. Several children met the criteria for discontinuation. The test took between 10 and 20 minutes to administer. All responses were scored according to the standard single-word scoring key presented in the BNT response booklet. Multiple possible names for an item would make it difficult to score an item reliably so we developed a list of acceptable answers to score the test in order to reduce ambiguity in scoring. Credit was given for a correct answer even if it was in another language. All scoring was checked by the assessor who administered the test and by a second assessor. Any disagreements were resolved through discussions. A score of ‘1’ was awarded for all correct responses. The final score was calculated by summing the number of spontaneously correct items and the number of correct items following a stimulus cue. The maximum possible score was 61.
Test administration. The expressive vocabulary test was administered as part of a
40
neuropsychological battery to a representative sample of 308 children. The battery included modified measures of various constructs including executive function, problem-solving, planning ability, verbal and working memory, attention, cognitive flexibility and non-verbal reasoning. The tests are described in detail in Chapter 3.
Other measures. In the test of non-verbal reasoning (CPM), the child was presented with a matrix of abstract patterns with one missing piece - six pieces with various patterns appeared below the matrix. Only one piece completed the missing pattern in the matrix. The child was required to complete each matrix by placing the correct piece in the empty space. A demonstration trial was administered where the assessor showed the child which of the six pieces completed the picture on the first sheet. For the next four matrices, if the child picked the wrong piece, the assessor explained why it was incorrect and then administered a second trial. The remainder of the items were presented only once. If the child did not get any of the items correct on the first set (Set A), the assessor stopped the test. All the children completed the first set. Three sets (A, B and C) with 12 matrices each were presented. The child’s score was the total number correct.
We also administered tests of reading (letters, words and sentences) and arithmetic (written and oral) (Bhargava, Bundy, Jukes, & Sachs, 2001) to 145 children in our study sample in order to quantify school achievement. In the reading tasks, children were required to indicate their choice of real letters, words and sentences from a list which included fake letters, words and sentences. The arithmetic tests required the child to complete addition, subtraction, multiplication and division items as well as answer some questions orally. To obtain the school achievement score, we summed the scores across each of the tests.
We obtained information on sociodemographic characteristics using a structured interview form. Based on a composite index of maternal and paternal education and occupation, ownership of livestock and type of household windows, children were assigned to one of three categories of household resources (Level 1, Level 2 or Level 3). The development of this index is described in detail in Chapter 3.
Analysis
A descriptive analysis of the background characteristics, distribution of scores and pattern of response was conducted. An independent samples t-test was conducted to compare naming performance on the initial and final versions of the KNT. Item difficulty, defined as the percentage of correct responses for each item, was assessed to determine whether the items included on the test had appropriate difficulty levels (easy, medium, hard). Internal consistency reliability of the KNT was measured by Cronbach’s alpha. Pearson product-moment correlation coefficients were computed to examine the relationship between the KNT and specific background variables (age, gender, school experience, area of residence, nutritional status and household resources) and between the KNT and measures of non-verbal reasoning (CPM) and
41
school achievement (reading and arithmetic). Mean differences in performance according to the background variables were assessed using independent samples t-tests and univariate analysis.
Effect sizes were computed when background variables showed a significant association with child outcome. Regression analyses were conducted to determine the proportion of variance in naming performance accounted for by each of the background variables.
Results Descriptives
A description of the study sample is presented in Table 1. Children who completed the initial version of the KNT had significantly higher scores (M= 35.03, SD = 5.382; range 21-49;
N = 100) than those who completed the final version (M= 20.74, SD = 8.368; range 1-45; N = 208); t(267unequal variances) = 15.376, p<.001. To correct for the differences in difficulty levels, the scores were standardised thereby enabling direct comparison of the scores. The z-scores of all children were normally distributed.
The total number of correct responses was counted for each of the items (Table 2).
Overall, 59 out of 61 (97%) test items were named correctly by at least one child. An analysis of the pattern of responses demonstrated that on 20 items, less than 10% of the sample provided a correct response; on 28 items, between 10% and 62% of the sample provided a correct response while on 13 items, 63% and more of the sample answered correctly. All children accurately named 8 of the 61 items. None of the children responded correctly on 2 items. The KNT had a mean difficulty level of 0.58 (SD = 0.37).
Reliability and Validity
The KNT had an internal consistency reliability of .905 and a test-retest reliability level of .918. There was a weak correlation between the KNT and the school achievement score, r = .249 (p = .003). A moderate correlation, r =.464 (p < .001) was recorded between the vocabulary scores (KNT) and non-verbal reasoning (CPM).
Associations with Background Characteristics
Univariate analysis of variance revealed significant differences in performance on the KNT according to various background factors (Table 3). There was a significant effect of gender with boys performing better than girls. Younger children had lower performance levels than those who were older. Children with growth retardation performed more poorly than those who did not. There was a tendency towards higher scores for those who had more schooling experience than those with less. The results indicated that there were no significant differences in performance according to household resources and area of residence.
Multivariate Analysis
Significant correlations among background variables were generally small and ranged from -.300 to .427 (Table 4). Results from the regression analysis indicated that the overall model was statistically significant (F = 12.521, p = .000). Age, gender and school experience