KRIVET Issue Brief People are Our Hope
Changes in Education Equity Analyzed Using Programme for International Student Assessment (PISA) Data
- In 2018, compared with 2006, the academic “ resiliency ratio ” of the top 25% increased by 0.66% for the Organisation for Economic Co-operation and Development (OECD) average and decreased by 1.78% for Korea.
• This means that, in Korea, the economic, social, and cultural status (ESCS) of parents had a greater effect on students ’ grades than it did in other OECD countries.
- The “ resiliency ratio ” of the top 4%, which is the highest group, decreased overall in Korea as well as in OECD countries and other countries analyzed (USA, Japan).
• Is inequality handed down from parents to children? Yes.
• Does education break that loop? No. Rather, it is suspected to be a catalyst for widening the educational gap.
• Is inequality deeper than it was in the past? Yes.
- The solution is to develop a society in which parents ’ economic, social, and cultural status does not have a significant influence on student performance ― a society that can improve social advancement through the introduction of an educational system to overcome disadvantages in economic, social, and cultural status. To put it more simply, a radical change in the education system is necessary to create a society that can support movement from “ rags to riches. ”
01 Necessity for Data Analysis
| Analysis background: Is inequality handed down from parents to children? Does education break that loop? Is inequality deeper than it was in the past?
In his study of the degree of intergenerational mobility in the USA, Miles Corak described the “American Dream” as the belief that “individual talent, energy, and motivation determine outcomes and accomplishments, as opposed to family background and status”1. It should be questioned if this is indeed true in all countries.
- OECD publishes the “resiliency ratio,” which, based on PISA data, represents how disadvantaged students beat the socioeconomic odds stacked against them2. According to the OECD, students with poor PISA scores have poor socioeconomic backgrounds, but some of them (the percentage varies by country) show high academic performance despite the disadvantages they face.
- By the comparable standard of the PISA as of 2015, the proportion of top 25% performers, among all PISA participants that belong to the bottom 25% in terms of socioeconomic status (ESCS) of the parents, is considered the social and economic
“resiliency ratio.”
- If educational equity is to be recognized as a strategy3 that allows intergenerational mobility in a society, PISA’s “resiliency ratio”
among 15-year-olds can serve as a yardstick to measure “educational equity” to a degree.
- Theoretically, if the socioeconomic background of the parents does not have any effect on the academic performance of the students, any group of students with a certain level of grades—top 25% or 4%—should have the same proportion of parents in the ESCS quartile (25%). This has been used to create a new method of calculating the “resiliency ratio.”
I Footnote I
1) Inequality from generation to generation: the United States in Comparison, Miles Corak, 2012 2) PISA, the OECD”s Program
for International Student Assessment, is conducted every three years to evaluate reading, math, and science abilities of 15-year-old students in order to provide basic data for international comparative study of academic achievement and establishment of education policies in each country.
3) The strategies to find measures to enhance intergenerational mobility in education although the influence of educational outcome on that mobility can vary by country due to the difference in labor markets and welfare policies.
Publisher: Young Sun Ra | Date of issue: May 11, 2020 | Issued by: Korea Research Institute for Vocational Education and Training (KRIVET)
2020
No.184I Note I
This brief was excerpted and reorganized from Hwang, Seong-soo et al., Changes in Education Equity Analyzed Using PISA Data.
KRIVET Issue Brief
| Data for analysis: OECD, PISA 2006 and 2018 data
Subjects for analysis: 15-year-old students in Korea, the USA, Japan, and other OECD countries that participated in PISA 2006 and 2018
Variables for analysis
- The ESCS of parents: PISA’s yearly ESCS index was calculated and standardized based on the jobs and educational attainment of the parents and on household assets. To ensure representativeness, OECD final student weight (W_FSTUWT)4 was applied for all samples.
- Performance: PISA 2006 and 2018 data provide five and ten plausible values for each subject, respectively. Instead of observed values, PISA data presents arbitrary achievable scores, extracted from the posterior distribution estimating the ability scores for individual students (Jeong, Hye-gyeong & Jo, Ji-min, 2013). In this study, the average value of all plausible values for each student was used as the grade, and the final student weight of the OECD was applied. Considering the limitations of the individual students’ uses of plausible values, a chi-square test was conducted to see if the resiliency ratio differed between the two time points.
02 The change and limitation of resiliency ratio of OECD
| Korea’s “resiliency ratio” for science subjects by the OECD standards decreased by 3.21%p from 2006 to 2015, while Japan and the USA showed increases by 8.24%p and 12.28%p, respectively.
The OECD average increased by 1.51%p.
The resiliency ratio of science subjects, which is the main area of the PISA data published by the OECD, was in the following order in 2006: Korea > Japan > Korea > OECD average > USA. In 2015, the order changed to Japan > Korea > USA > OECD average.
- As for the changes in the ratio by country, Korea’s decreased from 43.56% → 40.36%. Contrarily, those of Japan and the USA increased from 40.55% → 48.79% and 19.34% → 31.62%, respectively, and the OECD average also increased from 27.67% → 29.18%.
[Figure 1] Resiliency Ratio Changes among OECD Nations (2006, 2015 science performance)
| The OECD resiliency ratio has a defect in that it is not easy to compare countries or time points, as this may reflect the economic power or institutional changes of a country.
As seen in Figure 1, the resiliency ratio of the PISA (the proportion of students who performed among the top 25% out of the bottom 25% of students by parental socioeconomic status by country) until 2015, has the following limitations.
- First, the resiliency ratio can depend heavily upon the national power, including the economic power, of a country. For example, if the ESCS of the parents in the bottom 25% of a rich country is higher than that of the top 25% of a poor country, it becomes difficult to explain who was more disadvantaged between the two countries and how well they have overcome those disadvantages.
- Second, this is a concept of resilience at a certain point of time and, thus, is difficult to compare to other points of time, which is required to explain causal relations. For instance, even if the “resiliency ratio” of a country was 50% at a certain point of time and decreased to 30% at the next point, this does not naturally indicate that educational equity has worsened in the country. This is because the economic power or quality of public education of a country is variable, not constant. To put it more simply, even if the academic performance of a specific country remains the same, if many students in other countries overcome disadvantages because their parents’ socioeconomic status or the public education system has improved over a certain period of time, this change may make educational equity of the consistent country seem relatively deteriorated.
I Footnote I
4) Final student weight includes school and student weights and corrections for schools and students not attending the assessment (OECD, 2012:
121).
I Note I
1) The ratio of students who overcome disadvantages, as published by the OECD, is based on the result of evaluation in main areas for each year.
2) The dotted line indicates that the ratio difference is not significant, and the solid line indicates significant difference at the 1% significance level.
Source: OECD OECD average Korea Japan USA
50.0 45.0 40.0 35.0 30.0 25.0 20.0 15.0 10.0
2006 2015
43.56
48.79 40.36 31.62 29.18 40.55
27.67
19.34
(%)
May 11, 2020
- To overcome this limitation, the OECD criteria was changed in 2018 to the percentage of students who score within the top 25%
of each country’s grades and whose parents belong to the bottom 25% in terms of their socioeconomic status. Therefore, the data published in 2018 cannot be compared in time series with the past data. In addition, as the OECD only publishes the ratio of the major subjects for each survey, time series comparison of the same subject is only possible at nine-year intervals. This means unless the current OECD publication method changes, the reading scores of 2018 cannot be compared in time series until 2027.
| It is critical to develop a new “resiliency ratio” based on data comparable to past PISA results.
In this study, we complemented the existing method by comparing the parents’ ESCS and the grades within a country, which allows for comparison of educational equity between countries.
- This “new resiliency ratio,” which is not influenced by differing educational excellency5 or economic power or by the overall social conditions of a country, enables the identification of the proportion of disadvantaged students who overcome the odds in a country and makes comparison between countries and points of time possible. .
03 Comparison Between Time Points of the “ New Resiliency Ratio ” by Subject Based on the KRIVET Standards
| The new resiliency ratio, which allows for the comparison of educational equity at the individual national level, is defined as the ratio of the bottom 25% of students in terms of parental socioeconomic status who are in the top 25% for educational achievement in a country.
The proportion of students whose parents belong to the bottom 25% in terms of socioeconomic status out of the top 25%
of students in Korea was compared between 2006 and 2018.
- Compared with 2006, the resiliency ratio in 2018 increased by 0.66% on average in OECD countries and decreased by 1.78% in Korea. This means that, in Korea, the socioeconomic background of parents has a greater influence on student performance than in other OECD countries.
- For overall grades in reading, math, and science, Korea’s “resiliency ratio” was about 13.46% in 2006 but decreased significantly to 11.68% in 2018. This implies that educational equity in Korea decreased in 2018 from the previous reference point in 2006.
- The resiliency ratio in Korea decreased by 2.32% in reading, 1.23% in mathematics, and 0.75% in science.
- Similarly, Japan has seen a declining trend in resiliency ratio, while the USA has seen a rise in resiliency ratio for all subjects.
However, educational equity in the USA is still significantly lower than the average of OECD nations as well as major countries.
[Figure 2] New Resiliency Ratio by Subject: the Ratio of the Bottom 25% of Students in Terms of Parental Socioeconomic Status out of the Top 25% of Students
| The new resiliency ratio to compare educational equity at an individual national level is defined as: the ratio of the bottom 25% of students in terms of parental socioeconomic status in the top 4%
of students.
The top 25% of academic performance, as previously suggested, was to match the standard of the resiliency ratio by PISA.
In contrast, the top 4% standard was drawn from the first grade of Korean school grade standard and used to examine the I Footnote I
5) This is a comparative advantage. That is, the educational achievement of a country is superior to that of other countries.
I Note I
1) PISA 2006 data does not provide reading scores for the 2) The dotted line indicates that USA.
the ratio difference is not significant and the solid line indicates significance at the 1% significance level.
OECD average OECD average Korea Japan USA USA
16.0 15.0 14.0 13.0 12.0 11.0 10.0 9.0 8.0
7.0 2006 2018
13.46
11.68 9.919.89 8.88 11.85
9.25 7.78
All
16.0 15.0 14.0 13.0 12.0 11.0 10.0 9.0 8.0
7.0 2006 2018
12.02
10.79 9.759.39 7.97 11.08
9.72 7.95
Mathematics
16.0 15.0 14.0 13.0 12.0 11.0 10.0 9.0 8.0
7.0 2006 2018
15.29
12.97 10.86 10.54 9.57 11.99
10.93
Reading
16.0 15.0 14.0 13.0 12.0 11.0 10.0 9.0 8.0
7.0 2006 2018
14.13
13.38
10.95 10.59 9.46 13.22
10.04 8.56
Science
(%)
(%)
(%)
(%)
KRIVET Issue Brief
｜ KRIVET Social Policy Building, Sejong National Research Complex, 370, Sicheong-daero, Sejong-si, Republic of Korea ｜ Tel: 044-415-5000/5100 ｜ www.krivet.re.kr ｜ ratio of students whose parents’ socioeconomic status was in the bottom 25%.
- In 2006 and 2018, Korea showed a higher resiliency ratio (based on the top 4%) for all subjects than did Japan and the USA, but all three countries, as well as the OECD average, showed an overall decline in the resiliency ratio.6
- The fact that most countries, including member countries in the OECD, are witnessing a declining trend in the resiliency ratio (based on the top 4%) means that the probability of the top students having parents with low social and economic status is becoming significantly lower. In short, the result indicates that we are gradually moving away from a “resilient society.”
[Figure 3] New Resiliency Ratio by Subject: The Ratio of the Bottom 25% of Students in Terms of Parental Socioeconomic Status out of the Top 4% of Students
04 mplications
Implications of the PISA Analysis
- Is inequality handed down from parents to children? Yes.
- Does education break that loop? No. Rather, it is suspected to be a catalyst for widening the educational gap.
- Is inequality deeper than it was in the past? Yes.
In a 2012 study, Miles Corak said that, without changes in these underlying factors, the transmission of inequality from the current generation to the next will remain the same as in past generations.
In general, students with better socioeconomic backgrounds tend to be healthier, have better school grades, and perform better in the labor market. Better grades, gained despite the disadvantages, can increase social mobility (PISA, 2006); thus, education can improve the living conditions for future generations, as well as provide better life opportunities for individuals.
We need fundamental changes to the educational system in order to create a society in which household socioeconomic status does not significantly affect the students’ academic performance. Social mobility can be increased through the introduction of an educational system that allows students to overcome the disadvantages of their socioeconomic background—a “resilient society.”
Hwang, Sungsu (Ph.D., Research Fellow, KRIVET) Lee, Jieun (Senior Research Associate, KRIVET) Keum, Yejin (Research Associate, KRIVET)
I Footnote I
6) Korea’s resiliency ratio, by subject, decreased in reading (2.65%), mathematics (2.08%), science (5.95%), and in all subjects (1.11%).
In particular, the decrease in the resiliency ratio in s c i e n c e s u b j e c t s w a s large. This means that the socioeconomic status of parents has a greater influence than it did in the past.
I Note I
1) PISA 2006 data does not provide reading scores for the USA.
2) The dotted line indicates that the ratio difference is not significant and the solid line indicates significance at the 1% significance level.
OECD average Korea Japan Japan USA
(%)
(%)
(%)
(%)
12.0 10.0 8.0 6.0 4.0 2.0
0.0 2006 2018
7.50
6.39 3.673.58 1.45 3.97
3.14
All
12.0 10.0 8.0 6.0 4.0 2.0
0.0 2006 2018
7.43
5.35 4.11 3.04 1.74 4.634.59
2.64
Mathematics
12.0 10.0 8.0 6.0 4.0 2.0
0.0 2006 2018
10.65
8.00
4.844.53 3.32 5.514.93
Reading
12.0 10.0 8.0 6.0 4.0 2.0
0.0 2006 2018
12.09
6.14 4.844.02
1.05 4.624.73
3.27
Science 3.93
