STANDARDS-BASED ASSESSMENT
Objectives:: After reading this chapter, you will be able to:
• Understand the crucial role of standards in educational instruction and assessment, especially in standardized testing
• Examine a set of standards for a speci fied age, level, and context and apply them to contexts of your own
• Analyze the purpose, advantages, and disadvantages of standards-based assessment
• Apply principles of standardization to the construction of teacher-based standards
• Appreciate the two-edged sword of large-scale standards-based testing—its social, political, and ideological consequences
• Be prepared to take action in your own teaching and assessing to ensure fairness and openness for your students
Throughout history, institutions of all kinds have required testing to confirm capabilities or qualifications. Some of the earliest formal examinations or tests have been traced back almost 2000 years to the Han Dynasty in China, where they were used to select the highest officials in the country (Cheng, 2008a). In the Bible (Judges 12:5-6), we read about the even earlier use of the so-called shibboleth test, by means of which two ethnically and linguistically different groups of people were distinguished. When an Ephraimite was ordered to pro
nounce the word shibboleth, his language caused him to say “sibboleth,” with an /s/, as opposed to the Gileadites’ pronunciation that used a /sh/. The conse
quences were dire: Ephraimites were thus exposed as aliens attempting to enter Gilead and were put to death. In another example, before World War II, the Australian government used language tests as a method to ban immigrants from other countries (T. McNamara, 2000). A government officer could select any language for the dictation test that the immigrant had to take.
Today, we are all still deeply affected by tests and examinations, especially high-stakes standardized tests. For almost a century, schools, universities, busi
nesses, and governments have looked to standardized measures for economical, reliable, and valid assessments of those who would enter, continue in, or exit their institutions. Proponents of these large-scale instruments make strong claims for their usefulness when great numbers of people must be measured 90
quickly and effectively. Those claims are well supported by reams of research data that comprise construct validations of their efficacy and the specification of standards or benchmarks that are to be incorporated into assessment instru
ments. Vie have become a world that abides by the results of standardized tests as if they were sacrosanct, having been blessed by research findings and insti
tutional standards.
In this chapter, we look at wha t standards underlie many large-scale stan
dardized tests, where they come from, and whether their validity is sound. Our purpose is to raise your awareness of standards-based assessment—measures that are used to evaluate student academic achievement and show that students have reached certain performance levels. With this backdrop, we then turn in Chapter 5 to issues surrounding the standardized tests that such standards are intended to support.
THE ROLE OF STANDARDS IN STANDARDIZED TESTS
Ask non--language specialists what a standardized test is and they are likely to tell you it’s a multiple-choice test, and then they will give you an example, such as the SAT® or GRE®. By now you know that this is not a complete answer. A standardized test presupposes, among other things, certain standard objectives or performance levels—now better known as standards (and also known as benchmarks)—that are held constant across one form of a test to another. The standards that underlie standardized tests are usually a set of carefully defined competencies that apply to a course, a curriculum, a year-long program, or even multiple-year objectives for, say, a K-12 program or secondary school gradua
tion criteria. Standards-based assessment refers to procedures that are specifi
cally designed to test such competencies.
Where do these standards come from? Who designs them? How are they incorporated into assessment instruments? The past 30 years have seen a mush
rooming of efforts on the part of educational leaders worldwide to base the plethora of school-administered standardized tests on clearly specified criteria within each content area measured For example, most departments of educa
tion at the state level in the United States have now specified the appropriate standards (that is, criteria or objectives) for each grade level (kindergarten through grade 12) and each content area (math, language, sciences, arts).
The construction of such standa rds makes possible a concordance between standardized test specifications and the goals and objectives of educational pro
grams. Educational reform goals such as the implementation of standards are efforts to improve education and raise the achievement of all students. By care
fully examining existing curricular goals, conducting needs assessments among students, and designing appropriate assessments of those standards, educators have sought to pinpoint desired educational outcomes for students. The intent is for these standards to serve as ¡guidelines for curriculum, assessment, and
instructional design that correspond to what students should know and be able to do as they progress through school, thus enhancing the teacher’s knowledge of student achievement and goals.
STANDARDS-BASED EDUCATION
A number of countries have implemented standards-based education, according to a report published in 1993 by the National Education Standards and Improve
ment Council. For instance, in China, the State Education Commission in Beijing sets standards for the entire country and for all levels of the school system. In England, standards-setting was considered the responsibility of local schools, but in 1988, the Education Reform Act mandated and outlined the process for estab
lishing a national curriculum. Similarly, Japan has created a system of national standards commonly called the Course of Study (Guidelines) through their Min
istry of Education's official curriculum (www.mext.go.jp/english/), which sets standards for the content of instruction in schools and then administers large
scale examinations to test attainment of those standards (Nakayasu, 2016).
In many countries, standards are developed specifically for language learn
ing. For example, a goal for making foreign language instruction more commu
nicative in Europe is presented in the formulation of a set of standards known as the Common European Framework of Reference (CEFR) for Languages (Council of Europe, 2001). Rather than calling it a standard or benchmark, the term framework is used because it “provides a common basis for the elabora
tion of language syllabuses, curriculum guidelines, examinations, textbooks, etc. across Europe” (p. 1). Initially, these standards were for transferring lan
guage proficiency credentials across national borders in Europe so that those who need to show language proficiency, such as migrant workers and profes
sionals, could be recognized. However, the Council of Europe saw language use as a broader ability, calling it plurilingual competence, in which all knowledge and experience of language and culture interrelate and interact For example, in different situations, an individual can easily call on different aspects of their linguistic repertoire to communicate effectively with another person.
Today CEFR policy goals are well established in European education at all levels. Administrators, course designers, teachers, and examining bodies are called on to reflect on their current practice and to ensure they meet the needs of their learners. Most noteworthy is that in addition to the standards that describe communicative competencies (language proficiency levels Al and A2, Bl and B2, Cl and C2), CEFR’s illustrative “can do” descriptors are available to learners for self-assessment and for documenting continued language learning progress. Given the prominent role of CEFR, the European Association for Lan
guage Testing and Assessment has established a special interest group to address issues related to the use and further development of the CEFR
In the United States, with its millions of nonnative English speakers, one particular challenge in constructing standards over the past couple of decades
has been in language arts education. ESL, also known as English for speakers of other languages (ESOL), English language learners (ELLs), and English language development (ELD), have become household terms for elementary and second
ary school teachers. The standards movement in the United States has placed a strong emphasis on educational equity. Not only are standards intended to make educational expectations clear and measurable, they also set high expectations for all students—including students who are second language users.
Although the number of schoolchildren from linguistically and culturally diverse backgrounds enrolled in U.S. schools has grown markedly, ESL was for many years not a federally designated content area for standards development.
To ensure that ELLs would have access to effective educational programs and the opportunity to reach high standards, the Teachers of English to Speakers of Other Languages (TESOL) organization developed ESL standards for education (Gottlieb, Carnuccio, Ernst-Slavit, & Katz, 2006; WIDA Consortium, 2012). The standards state what students shoul d know and be able to do as a result of ESL instruction and set goals for students’ social and academic language develop
ment and sociocultural competence. These ESL standards take a functional approach to language learning and use and allow for maximum flexibility in curriculum and program design. Table 4.1 lists the nine ESL content standards, organized under three educational goals.
Table 4.1 ESL Standards
Goals Standards
1. To use English to communicate in social settings
a. Students will use English to participate in social interactions.
b. Students will interact in, through, and with spoken and written English for personal expression and enjoyment.
c. Students will use learning strategies to extend their communicative competence.
2. To use English to achieve academically in all content areas
a. Students will use English to interact in the classroom.
b. Students will use English to obtain, process, construct, and provide subject matter information in spoken and written form.
c. Students will use appropriate learning strategies to construct and apply academic knowledge.
3. To use English in socially and culturally appropriate ways
a. Students will use appropriate language variety, register, and genre according to audience, purpose, and setting.
b. Students will use nonverbal communication appropriate to audience, purpose, and setting.
c. Students will use appropriate learning strategies to extend their sociolinguistic and sociocultural competence.
From Short (2000).
The TESOL ESL standards, although quite useful, did not provide sufficient details to educators who needed to assess ELLs in content areas. World-Class Instructional Design and Assessment, a consortium of a number of U.S. states, stepped in to provide benchmarks that bridged the gap between language learner standards and standards for all learners. In 2001, the U.S. federal gov
ernment entered the usually state-governed setting of standards with the now infamous No Child Left Behind Act, which set into motion the development of new state standards in an attempt to close the achievement gap among all stu
dents. Despite its good intentions, No Child Left Behind left teachers and admin
istrators in a quagmire of political, economic, educational, and assessment wrangling (Blake, 2008; Meier & Wood, 2006).
In 2010, a number of states adopted the same standards for English and math and, in 2011, for science (Next Generation Science Standards). Fre
quently called the Common Core State Standards (CCSS), these standards help all students to pursue common educational goals, even, if they change schools or move to a different state. As with other educational reforms, the CCSS have drawn both support and adverse criticism from politicians, ana
lysts, and commentators. As standards-based education in the United States has prompted numerous debates, English language proficiency (ELP) devel
opment standards for students learning English has been the prerogative of each state to determine. In 2012, the Council of Chief State School Officers provided the ELP development framework to assist states in revising their ELL standards to correspond to the CCSS (TESOL International Association, 2013). Language education reform is such a hot-button issue that research journals devote whole issues to the topic. (See TESOL Quarterly 2014, vol. 48, issue 1.)
DESIGNING ENGLISH LANGUAGE STANDARDS
The process of designing and conducting appropriate periodic reviews of Eng
lish language standards involves dozens of curriculum and assessment special
ists, teachers, and researchers (Bailey, Butler, & Sato, 2007; Bailey & Carrol, 2015; Bailey & Wolf, 2012).
In creating such “benchmarks for accountability" (O’Malley & Valdez Pierce, 1996), standards designers have a responsibility to carry out a comprehensive study of a number of domains:
• literally thousands of categories of language, ranging from phonology at one end of a continuum to pragmatic, functional, and sociolinguistic elements at the other end
• specification of what ELD students’ needs are, at 13 different grade lev
els, to succeed in their academic and social development
• a consideration of what is a realistic number and scope of standards to be included within a given curriculum
• a separate set of standards (qualifications, expertise, training) for teach
ers to teach ELD students successfully in their classrooms
• a thorough analysis of the means available to assess student attainment of those standards
We already noted that standards-setting for English language courses is a global challenge. In many non-English-speaking countries, English is now a required subject starting as early as the first grade in some countries and by the seventh grade in virtually every country. With the worldwide increase in demand for English, a “communicative” curriculum in English is often required during the early elementary school grades. Such mandates from ministries of education require the specification of standards, or benchmarks, on which to base curricu
lar objectives. Sometimes such standards are not reasonable because of the English proficiency of teachers and the practical uses for English in the real world outside the classroom (Akiyama, 2004; Byun et al., 2011; Chinen, 2000;
Hu & McKay, 2012; M. Sakamoto, 2012; Yoshida, 2001).
In Japan, the lack of spoken English skills prompted the Ministry of Educa
tion to include an oral skills component in their senior high school exit exami
nation. However, these efforts revealed competing social, cultural, and educational values of the stakeholders affected by the examination and raised questions about the actual purpose of the assessment and its validity in terms of the curriculum (Akiyama, 2004; Hashimoto, 2013). In response to some of the concerns noted above, Japan has been researching a new' Test of English for Academic Purposes to be used as a benchmark for university entrance (Green, 2014; Taylor, 2014; Weir, 2014).
Australia has also developed common standards and benchmarks in Eng
lish language and literacy and acknowledges the conflicting motivations in their development—education, assessment, and accountability (Davison &
McKay, 2002; Hammond, 2014; McKay & Brindley, 2007)—that result in differ
ent assumptions by stakeholder groups. In the United States, the goals of the state of California’s English Language Development Standards are described as follows:
ELs must have full access to high-quality English language arts, mathemat
ics, science, and social studies content, as well as other subjects, at the same time that they are progressing through the ELD-level continuum. The CA ELD Standards correspond with the CA CCSS for ELA/Literacy and are designed to apply to English language and literacy skills across all academic content areas, in addition to classes specifically designed for English language de
velopment. The CA CCSS for ELA/Literacy raise expectations for all students in California. Among other things, students are expected to participate in sustained dialogue on a variety of top ics and content areas; explain their thinking and build on others’ ideas; construct arguments and justify their positions persuasively with sound evidence; and effectively produce written and oral texts in a variety of informational and literary text types. ELs must
successfully engage in these challenging academic activities while simultane
ously developing proficiency in advanced English. (California Department of Education, 2014)
The chart in Figure 4.1 shows proficiency descriptors for the state of California for ESL across several levels. These descriptors provide a sense of how standards are stated in broad terms, but as they become more age- and subject matter-specific, they are of course more detailed. Hundreds of much more specific standards are listed in the publication where these standards appear. As children’s intellectual development increases, metalinguistic aware
ness and grammatical/phonological accuracy are also broken down into spe
cific standards.
Assessing the academic achievement of every student is an essential part of educational reform, but one that presents a challenge for most schools, school districts, states, and countries. Further, in addition to the careful specification and devel opment of standards, educators also need to design assessment instru
ments that align w ith the standards.
STANDARDS-BASED ASSESSMENT
The development of standards obviously implies the responsibility for correctly assessing their attainment. As standards-based education became more accepted in the 1990s, many educational systems around the world found that the stan
dardized tests of past decades were not in line with newly developed standards.
This was the impetus to begin the interactive process not only to develop stan
dards but also to create standards-based assessments. The comprehensive pro
cess of developing such assessment in California still continues as curriculum and assessment specialists design, revise, and validate numerous tests (Hauck, Wolf, & Mislevy, 2013; Linquanti & Hakuta 2012; Menken, Hudson, & Leung 2014; Wolf, Guzman-Orth, & Hauck, 2016).
In California, the current state-required test for ELP assessment is the California English Language Development Test, a battery of instruments designed to assess the attainment of ELD standards across grade levels. However, this test is being replaced by the English Language Proficiency Assessments for California (ELPAC), which is aligned with the 2012 California English Language Development Stan
dards, which are in turn aligned with the California Common Core Standards for English Language Arts. Because the ELPAC is new, the process of adminis
tering a comprehensive, valid, and fair assessment of ELD students continues to be perfected.
Similar standards-based assessments exist elsewhere such as Hong Kong, China, where the School Based Assessment has been introduced as part of the Hong Kong Certificate of Education Examination (Qian, 2014). Across the globe
in countries such as South Africa, Brazil, Chile, and Poland—standards-based assessment is now' commonplace, with all the advantages and disadvantages that
