The CMS was standardized on a representative U.S. sample of 1000 children.
The sample was stratified according to age (10 age groups ranging from 5 to 16 years of age), sex (equal number of males and females), race/ethnicity (White, African American, Hispanic, and Other), geographic region (north- east, north central, south, west), and parent education (5 categories ranging from less than 8th grade to university degree).
Reliability coefficients were calculated on the standardization data: split- half and test-retest. As reported in the CMS Manual, average split-half relia- bility estimates for the subtests ranged from .71 (Faces Immediate) to .91 (Word Pairs Learning). For the indexes, split-half reliability estimates ranged from .76 (Visual Immediate and Visual Delayed) to .91 (General Memory).
Test-retest reliability was assessed using a subsample of 125 students across 3 age bands. The average reliability estimates across subtests ranged from .54 (Dot Locations) to .86 (Word Lists). Across each of the 3 age bands, test- retest reliability estimates for the indexes ranged from .29 (Visual Immedi- ateml 3 to 16 years age band) to .89 (Attention/Concentration--9 to 12 years age band).
Given the nature of the learning and memory construct, additional analy- ses were conducted to further assess score stability over time. Decision con- sistency reliability coefficients were calculated for each of the subtests, and index scores were calculated for each of the three age bands. Results indi- cated that a general practice effect of up to one standard deviation was ap- parent; this practice effect was similar to those on the WISC-III Performance IQ (Cohen, 1997). Interrater reliability was also assessed on those subtests requiring more subjective scoring (e.g., Stories, Family Pictures); interrater reliability was found to be high.
Content, construct, and concurrent validity were examined using the stan- dardization sample. Content reviews on the CMS test items were conducted by expert pediatric neuropsychologists and school psychologists. Construct validity was assessed using confirmatory factor analysis; results of the factor analysis provided strong support for the three-factor model: attention/con- centration and delayed subtests of the verbal and visual subtests. Concurrent validity studies were conducted with intelligence and achievement tests
(e.g., the Wechsler Intelligence Scale for ChildrennThird Edition [WISC-III], the Wechsler Individual Achievement Test [WIAT], and the Otis-Lennon School Ability Test [OLSAT]). Correlations between the CMS and these mea- sures indicated that the General Memory Index is moderately correlated with general, verbal, and nonverbal intellectual abilities. The Attention/Concen- tration Index correlated highly with other measures of complex attention.
Verbal intellectual ability correlated higher with auditory/verbal measures than with visual/nonverbal measures. CMS and measures of achievement correlated moderately; the strongest relationship was exhibited between the Attention/Concentration Index and total academic achievement.
CMS Subtests and S u b s c a l e s
The CMS is designed for children ages 5 through 16 years and includes the assessment of learning and memory in three areas: (a) auditory/visual learn- ing and memory (verbal), (b) visual/nonverbal learning and memory (visual), and (c) attention/concentration. Each of the three areas is measured by two core subtests and one supplemental subtest. Both the auditory/verbal and visual/nonverbal areas include subtests that have both an immediate and a delayed memory portion. Auditory/visual subtests include Stories, Word Pairs, Word Lists (supplemental); visual/nonverbal subtests include Dot Lo- cations, Faces, and Family Pictures (supplemental); and Attention/Concen- tration area is composed of the Numbers, Sequences, and Picture Locations (supplemental) subtests. (See Table 1.2.)
CMS Scores and Interpretation
All of the subtests are reported in scaled scores with a mean of 10 and a stan- dard deviation of 3. Index scores (mean - 100; standard deviation - 15) can be derived for each area/domain: auditory/verbal, visual/nonverbal, and at- tention/concentration. A General Memory Index score (mean - 100; stan- dard deviation - 15) that includes the immediate and delayed memory indexes from both the visual/nonverbal and auditory/verbal areas can also be calculated.
The steps recommended for the interpretation of the CMS are similar to the interpretive steps on the WISCqlI. The first step involves the interpreta- tion of the General Memory Index as a measure of global memory function- ing. The General Memory Index can be compared to the WISC-III FSIQ score as an indication "of whether or not a child's ability to learn and remember new material is commensurate with what one would expect based upon his/her intellectual potential" (Cohen, 1997, p. 156). The CMS Manual pro- vides tables for both predicted difference and simple difference methods. The CMS offers two methods of IQ~memory discrepancy analysis: the simple- difference method and the predicted-difference method. Although both methods are presented, the predicted-difference method is generally pre-
TABLE 1.2
CMS Subtests and Description
Domain Subtest Description
Auditory/Verbal Stories Word Pairs Word Lists Visual/Nonverbal Dot Locations
Attention/Concentration Faces Family Pictures Numbers Sequences Picture Locations
Ability to recall meaningful and semanti- cally related verbal information Ability to learn a list of word pairs over a
series of learning trials
Ability to learn a list of unrelated words over a series of learning trials Ability to learn the spatial location of an
array of dots over a series of learning trials Ability to remember and recognize a series
of faces
Ability to remember scenes of family members doing various activities Ability to repeat random digit sequences
of graduated length
Ability to manipulate and sequence auditory/verbal information rapidly Immediate visual/nonverbal memory for
spatial location of pictured objects source: From The Children's Memory Scale, by M. J. Cohen, 1997, San Antonio, TX: The Psychologi- cal Corporation.
ferred because of two primary considerations. The formula for the predicted- difference method not only takes into account the reliabilities and the corre- lations between the two measures but also corrects for regression to the mean. In general, FSIQ should be used as the best estimate of overall intel- lectual ability. If the difference between an examinee's VIQ and PIQ scores is 11-15 points or more (p ~ .05), however, the higher of these two IQ scores could be used instead of the FSIQ score as the best estimate of intellectual ability. Similarly, the General Memory Index score is usually the best esti- mate of a child's memory functioning. When there are meaningful differences between the measures that make up the General Memory Index, other CMS measures may be used.
The second step involves the interpretation of the Attention/Concentra- tion Index; a child's performance on this index should be compared to that on the WISC~III FSIQ using the predicted discrepancy model. A discrepancy that is deemed statistically and clinically significant should be examined fur- ther. For example, a difference of 25 points between the actual CMS Atten~
tion/Concentration Index and the predicted index score based on the WISC- III FSIQ is obtained by only 2% of children in the CMS-WISC-III linking sample. These results should be investigated further to determine the nature and extent of these attention difficulties across settings (e.g., home/school).
The third step involves the interpretation of the Verbal and Visual Memory Indexes; this step allows comparison between material that is learned and remembered aurally and material learned and remembered visually. To illus- trate, a 19-point difference or less between the Visual Immediate Index and the Verbal Immediate Index is found in about 31% of children in the CMS standardization sample. Put another way, almost one of every three children in the standardization sample had a discrepancy of 19 points or lower, sug- gesting that it is rather common to see verbal-visual differences of this mag- nitude in children. Comparisons can also be made between the child's abil- ity to hold material in working memory and the child's ability to store and retrieve learned material from delayed memory (i.e., Working Memory Index versus the visual/verbal immediate and delayed indexes).
Subtest interpretation is included in the fourth step. Similar to the WISC- III subtest interpretive format, the individual subtest scores can be compared to other scores within their respective indexes.
The fifth step includes the comparison between the Verbal Delayed Index and the Delayed Recognition Index. Discrepancies between the two indexes allow comparison between encoding/storage aspects of memory and re- trieval of material. For example, a large difference between the Verbal De- layed Index (a recall measure) and the Verbal Delayed Recognition Index may provide valuable clinical information. The retrieval of information via recall is more demanding than the recognition condition because the child is re- quired to impose an organizational structure on the stored information. The ability to access the previously learned information during the recognition condition is enhanced because the child is exposed to the actual material and is simply required to identify the information. If the child has signifi- cantly greater access to memories under the recognition condition than un- der the recall condition, the assumption is made that the child has actually learned the material but is unable to access the information under the more demanding recall process. This type of retrieval difficulty can be remediated by having the child provide self-cues and structure in order to facilitate remembering.
The sixth step involves the interpretation of the Learning Index. In order to interpret this index score, a large discrepancy between the predicted Learn- ing Index (based on the WISC-III FSIQ and the actual Learning Index--with the actual index being lower) may be indicative of a generalized learning impairment.
THE WISC~Iil AND THE CMS AND CLINICAL POPULATIONS
The WISC-III and, to a lesser extent, the CMS, have been administered to spe- cific clinical populations (e.g., learning disabled, attention~deficit/hyperac-
tivity disorder [ADHD]) in an effort to determine if there are subtest/factor profiles that define these individuals as a group (see Prifitera & Saklofske, 1998). To date, profile analysis has received mixed reviews; most researchers caution against such an analysis given the heterogeneity within each of these groups and the potential for misinterpretation based upon a single test ad- ministration (Daley & Nagle, 1996). On the other hand, this kind of analysis can enhance clinical decision making in determining strengths and weak- nesses. Children referred for psychological assessment often present with less than easily diagnosed conditions. Both subtest and factor profiles may serve a most important role in both a priori and a posteriori hypothesis test- ing, and eventually contribute to a diagnosis.
Research conducted with children with learning disabilities suggests that particular subtypes of learning disabilities in older children (e.g., nonver- bal learning disability; basic phonological processing disorder) are identifi- able using the WISC-III VIQ~PIQ discrepancies (Rourke, 1998). As well, stud- ies using the CMS with children with learning disabilities (e.g., phonological processing deficits) have demonstrated that these children exhibit mild to moderate impairments in attention/immediate working memory and audi- tory/verbal memory (Cohen, 1997).
Children with ADHD have been assessed with the WISC~III, resulting in mixed findings. Some research studies suggest that children with ADHD demonstrate significant deficits on the Freedom from Distractibility factor, a factor that measures attentional and memory processes (e.g., Prifitera &
Dersh, 1993; Schwean, Saklofske, Yackulic, & Quinn, 1993). Other studies, however, have not replicated these results (e.g., Semrud-Clikeman & Lorys- Vernon, 1988). Also, lower scores on the Processing Speed factor (a measure of the rapidity with which simple or routine information can be processed without making errors) relative to both the WISC-III FSIQ and the Perceptual Organization Index have been reported (Prifitera & Dersh, 1993; Schwean et al., 1993).
On the CMS, however, children with ADHD have been shown to demon- strate lower scores on the Attention/Concentration Index and the Verbal Im- mediate Index. A small percentage of children with ADHD (i.e., approximately
10%) may also demonstrate verbal memory dysfunction (Cohen, 1997).
CRITIQUE OF THE WISC~III AND THE CMS
The WISC-III and the CMS represent two measures in the assessment of cog- nitive processing that have been subjected to intense study and review. The diagnostic strength of these two instruments lies in the comparisons that can be drawn between the intellectual and memory components of cognitive pro- cessing. However, the diagnostic utility has been hampered in the past by the limitations regarding direct treatment outcomes. These criticisms have, in
part, been addressed by the reconceptualization of the WISC-III as a process instrument. An analysis of processing strengths and weaknesses is more amenable to skill remediation than the mere description of subtest scores.
The WISC-III continues to provide the basis for diagnostic decision mak~
ing; for example, a diagnosis of a learning disability is predicated upon a sig- nificant discrepancy between aptitude (e.g., IQ score) and achievement. AI- though the development of interventions is based upon both quantitative and qualitative information, funding and placement decisions continue to be based upon IQ and achievement composite scores. The WISC-III has proven to be a reliable and valid instrument for these purposes.
Both the WISC~III and the CMS require considerable clinical skill in order to be utilized most effectively. These instruments are complex and multi~
dimensional and do not lend themselves readily to direct interpretation. For example, the WISC-III subtests themselves assess multiple constructs. The clinician must have sufficient clinical skill in first identifying the constructs being measured and then comparing performance across subtests measuring comparable constructs within the WISC-III and between the WISC-III and the CMS. The standardized or norm-referenced comparisons provide a nome- thetic paradigm for test interpretation; however, the clinician must also be able to develop hypotheses regarding performance based upon idiographic considerations. Again, these kinds of comparisons require specialized skills and practice.
CASE STUDY
John is a 9-year-old male who was referred for assessment due to a history of academic difficulties. A review of John's developmental and medical history indicated that John was born full-term after a normal pregnancy. Develop- mental gross motor milestones were attained within normal limits; fine mo- tor development was slightly delayed. Developmental language milestones were also obtained within normal limits.
John's educational history indicated that he began experiencing read- ing difficulties in kindergarten. During the primary grades, John continued to have difficulty with reading; he demonstrated poor phonetic and word attack skills and displayed number and letter reversals. During grade 4, John's grades tended to be inconsistent across subjects. The teacher noted that John also had difficulty following spoken directions and appeared inattentive.
John was a willing participant during the testing sessions; his use of lan- guage was fluent and prosodic but was marked by mild word-finding diffi- culties. He also experienced difficulty expressing himself orally when the material was complex. Although his comprehension at the sentence level was good, he appeared to have difficulty retaining directions and had to repeat
them to himself frequently. In general, John's attention span, activity level, and impulsivity appeared to be age-appropriate within the context of the testing session. He displayed good motivation and task persistence through- out testing.
Among others, the following tests were administered during the session:
WISC-III, CMS, WIAT, and Woodcock-Johnson Tests of Achievement--Re*
vised. (See Table 1.3 for tests administered and accompanying scores.) Intel*
lectual functioning, as measured by the WISC-III, was found to be in the high average range. John obtained an FSIQ of 116 (CI = 110-121); VIQ of 112 and PIQ of 119. In general, John's verbal and visual*spatial reasoning abilities were uniformly developed. Analysis of factor/index scores revealed a relative strength in speed of processing and a relative weakness in focused auditory attention/working memory (VC = 113; PO = 113; FD = 96; PS = 126).
In order to assess learning and memory, John was administered the CMS.
Analysis of his performance on the subtests constituting the Attention/Con- centration Index indicated that John exhibited focused auditory attention/
working memory skills that were discrepant from his measured intellect. He also demonstrated reduced working memory for material presented orally.
WlSC*III IQ and factor scores
TABLE 1.3
S t u d e n t S c o r e s on A s s e s s m e n t P r o t o c o l s
WISCqlI subtest scores
FSIQ 116 Information 13 Picture Completion 11
VIQ 112 Similarities 14 Coding 16
PIQ 119 Arithmetic 11 Picture Arrangement 11
VC 113 Vocabulary 11 Block Design 12
PO 113 Comprehension 11 Object Assembly 14
FD 96 Digit Span 7 Symbol Search 14
PS 126
CMS Index scores
Visual Immediate 128
Visual Delayed 131
Verbal Immediate 112
Verbal Delayed 122
General Memory 133
Attention/Concentration 94
Learning ! 22
Delayed Recognition 97
Source: From case study data provided courtesy of Dr. M. J. Cohen.
Further, he had difficulty reciting the alphabet during the Sequences subtest.
Comparison of John's superior General Memory Index (GMI - 133) with his best estimate of intellectual potential (FSIQ -- 116) indicated that his ability to learn and remember was above expectancy. However, a more detailed analysis of John's performance indicated that he was demonstrating signifi- cant variability in his ability to learn and remember. Comparison of John's au- ditory/verbal and visual/nonverbal index scores indicated that his visual learning and memory were superior overall. In contrast, his verbal learning and memory ranged from average to superior. John demonstrated above av- erage ability to learn concise rote verbal material (Word Pairs) presented three times and superior ability to recall the material after a 30-minute delay.
However, his ability to learn lengthy verbal material presented once (Stories) was average. John had difficulty encoding material presented at the begin- ning of the paragraphs.
On the Woodcock-Johnson Tests of Achievement--Revised, John demon- strated low average phonetic word attack skills and average reading compre- hension. As a measure of written expression, John was administered the Writ- ten Expression subtest from the WIAT. John performed in the low average range on this subtest.
In summary, the results of the assessment indicated that John was cur- rently functioning within the high average range of intellectual ability. He demonstrated relative strengths in areas of visual-spatial perception/con- struction, sustained attention, and visual learning and memory. These were contrasted by relative weaknesses in phonological processing, focused audi- tory attention/working memory, and verbal learning and memory for lengthy material presented once. During testing, John was also noted to exhibit re- duced auditory working memory, having to repeat directions to himself be- fore completing tasks. Academically, John demonstrated relative strengths in arithmetic contrasted by relative weaknesses in basic reading and written expression that were of learning disability proportion. It is likely that the in- attention noted at home and school was due to his reduced auditory pro- cessing as opposed to an attention deficit disorder per se, given his aver- age performance in sustained attention for visual material. Taken together, this pattern of test performance was consistent with a diagnosis of Specific Learning Disability in Reading and Written Expression. As a result, it was rec- ommended that John receive special education services designed for chil- dren with learning disabilities. John's teachers were made aware that his word attack skills and knowledge of sight words were weak, given his meas- ured intellect. Because his visual learning and memory were superior and his phonological processing skills were poor, it was recommended that reading/writing instruction should emphasize visual and multisensory ap- proaches. Further, it was recommended that John would benefit from ap- proaches that relied heavily on tactile input and experiential learning. Given his learning disabilities and reduced auditory attention, a reduction in the