Teks penuh





Submitted to Biology Education Study Program in a Partial Fulfillment of the Requirements for the Degree of

Magister Pendidikan


RAJA NOVI ARISKA Registration Number: 8136173013




Raja Novi Ariska. Registration Number: 8136173013. The Roles of Biology Exercise Construction in Students’ Science Textbook, Students’ Sex, and School Level on PISA Test Achievement in Binjai. A Thesis. Biology Education Program. State University of Medan, 2015.

The aims of this study were to investigate the (1) construction of biology exercises items in students’ science textbook compared to PISA test items, (2) students’ sex, and (3) school level on PISA test achievement in Binjai, North Sumatera, Indonesia. The PISA items and biology items in students’ science textbook were purposively selected and analyzed by new Bloom’s taxonomy, knowledge dimensions and types of questions. Furthermore, a total of 695 students who age 15,3 to 16,2 years old from ten public schools were randomly selected and had tested with four different booklets of PISA biology test achievement, composed of 40 questions of 13 units. The result showed that test items construction in science textbook with 2013 curriculum was the most appropriate to PISA test items than the items within science textbooks of KTSP curriculum. In general, the students’ scientific literacy scored 48.92 ± 13.24 while PISA competencies scores respectively 46.77 ± 18.77 for identifying scientific issues, 49.48 ± 14.19 in explaining phenomena scientifically, and 46.30 ± 18. 69 for using scientific evidence. There was no difference of students’ sex to scientific literacy and the whole competencies. However, although the students in the same age 15 years old, the students who were one level higher, senior high school, had significant differences in competency of using scientific evidence than the student in junior high school (u =69,801.500; p = 0.000, whole respondents data; u = 5,405.000; p = 0.043, PISA operational age data). The differences in test item construction and period of learning science, partly, are the possible factors determine students’ achievement in PISA survey.



Penelitian ini bertujuan untuk menyelidiki peran dari: (1) konstruksi soal latihan biologi pada buku IPA siswa, (2) jenis kelamin, dan (3) jenjang sekolah terhadap prestasi tes PISA di Binjai. Soal-soal PISA dan soal-soal biologi pada buku IPA di ambil secara purposif dan dianalisis dengan menggunakan taksonomi Bloom edisi revisi, level kognitif dan dimensi pengetahuan serta format pertanyaan. Kemudian, sebanyak 695 siswa berusia 15,3 – 16,2 tahun dari sepuluh sekolah negeri di Binjai di ambil secara acak dan diuji dengan soal Biologi PISA yang disusun dalam 4 kode naskah soal, masing-masing terdiri dari 40 soal. Hasil penelitian menunjukkan bahwa konstruksi soal biologi dalam buku teks IPA dengan menggunakan kurikulum 2013 paling sesuai dengan konstruksi soal PISA dibandingkan dengan soal-soal biologi pada buku teks IPA dengan kurikulum tingkat satuan pendidikan (KTSP). Hasil tes terhadap soal-soal PISA pada konten biologi menunjukkan bahwa nilai rerata untuk kategori literasi sains adalah 48.92 ± 13.24 dan nilai rerata untuk masing- masing kompetensi yang dinilai PISA adalah 46.77 ± 18.77 untuk mengidentifikasi permasalahan ilmiah, 49.48 ± 14.19 dalam menjelaskan fenomena secara ilmiah, dan 46.30 ± 18. 69 dalam menggunakan bukti ilmiah. Uji Mann-Whitney menunjukkan tidak terdapat peran jenis kelamin terhadap kemampuan literasi sains dan tiga kompetensi PISA. Akan tetapi, walaupun berada pada jenjang umur yang sama (15,3 – 16,2 tahun) siswa yang berada pada satu tingkat lebih tinggi menunjukkan bahwa kemampuan siswa SMA secara signifikan berbeda dengan kemampuan siswa SMP dalam kompetensi menggunakan bukti ilmiah (U =69,801.500; P = 0.000, data seluruh responden; U = 5,405.000; P = 0.043, data responden dengan usia operasional PISA). Perbedaan konstruksi soal dan periode belajar IPA secara terpisah merupakan faktor-faktor yang mempengaruhi prestasi literasi sains siswa dalam survei PISA



Praise and sincerest gratitude be to the Almighty, Allah SWT, for his

immeasurable blessing and all of the kindness that ease her life, so that she finally

able to accomplish her thesis which entitled “The Role of Biology Exercises

Construction in Students’ Science Textbook, Students’ Sex, and School Level on PISA Test Achievement in Binjai”. It is intended as one of the requirement to obtain the degree of Magister Pendidikan from Biology Education Study

Program, Postgraduate Program State University of Medan.

The writer would like to express her special heartfelt thanks to the peoples

who never stop motivating, providing constructive advices, and commenting for

better improvement. Earnest gratitude is expressed for her thesis supervisor,

Syarifuddin, M.Sc., Ph.D. and Prof. Dr. rer.nat Binari Manurung, M.Si for their

invaluable time spent in giving guidance, suggestions, comments, and critics for

the improvement of this thesis.

She also owes debt to the peoples, Dr. Rahmad Husein, M.Ed, Dr.H.

Hasruddin, M.Pd., and Dr. Hj. Fauziyah Harahap, M.Si. as examiners, for their

constructive comments and suggestions for the improvement of this thesis. The

whole teachers and students who participated in this research at SMP Negeri 1

Binjai, SMP Negeri 2 Binjai, SMP Negeri 3 Binjai, SMP Negeri 7 Binjai, SMP

Negeri 11 Binjai, SMA Negeri 1 Binjai, SMA Negeri 2 Binjai, SMA Negeri 3

Binjai, SMA Negeri 4 Binjai, and SMA Negeri 5 Binjai.

Undying gratitude to her beloved parents, H. Raja Abidinsyah and Hj.

Aminatun Zuhriah for their boundless loves, attentions, encouragements, and

understanding every weaknesses of her. Especially, when she faced the puzzling

situations in the completion of this thesis, the amazing endorsement and deep

constructive advices always sheds to her. So she still could stay on the tract to

overcome it. Brothers and sisters, Raja M. Rizky B., Raja Suci R., Raja Nurul F.,

and Raja M. Akbar H. for their invaluable support when she conducted the


Thanks also to Pasca Dik Bio A 2013 for countless hours gathering

valuable feedback and the overwhelming friendship (Al Khudri, Amrullah, Elena,

Erlia, Dewi, Dina, Maidera, Jhonas, Mahpuzah, Sukmawati, Siska, and Dhian)

and relatives who cannot be mentioned more for the friendship, invaluable

support, patience, and insight.

May Allah always bless us. Finally, with all humility this paper hopefully

will be beneficial to contribute ideas in education, especially for the writer.

Medan, June 2015 The writer

Raja Novi Ariska



1.6.1. Theoretical Significance ……….. 11

1.6.2. Practical Significance ……….. 11

2.1.4. Textbook and Educational Achievement………. 15

2.1.5. Role of Textbook……….. 16

2.1.6. Scientific Literacy ……… 17

2.1.7. Scientific Learning in 2013 Curriculum ……….. 20

2.1.8. PISA………. 21 Focus of PISA……… 21 Dimension of PISA Scientific Literacy ... 23

2.1.9. Indonesian Achievement in PISA Survey…………...………. 27



3.3. Methods and Design of research ………. 38

3.4. Operational Definition……….. 39

3.4.1. Biology PISAQuestion………. 39

3.4.2. Biology Exercise Construction in Science Textbook……….. 39

3.4.3. Students’ Sex……… 40

3.4.4. School Level……… 40

3.4.5. PISA Test Achievement……….. 40

3.5. Research Instrument………. 41

3.5.1.Item Analysis and Result of Research Instrument ……….. 43

3.6. Research Procedure ………. 45

3.7. Data Analysis………... 49

3.8. Hypothesis ………... 53

CHAPTER IV. RESULT AND DISCUSSION 4.1. Result and Data Analysis………. 55

4.1.1.Distribution of Assessment Questions Based on Bloom New Taxonomy………. 55

4.1.2. Question Distribution Based on the Type of the Questions ………… 58

4.1.3. Students’ Scientific Literacy ……… 59

4.1.4. Students’ Responses to Each Item……… 66

4.1.5. Sex Differences to Scientific Literacy and Its Competencies ……… 67

4.1.6. School Level Differences to Scientific Literacy and Its Competencies 68 4.2. Discussion ……… 71

4.2.1.Distribution of Question Based on Bloom New taxonomy…………. 71

4.2.2.Distribution of Question Based on Type of Question……….. 74

4.2.3. Students’ Scientific Literacy……… 75

4.2.4. Students’ Responses to Items………... 79

4.2.5. Sex Differences to Scientific Literacy and Its Competencies ……… 103

4.2.6. School Level Differences to Scientific Literacy and Its Competencies 105




Table 2.1. Context for PISA Scientific Literacy Assessment………… 24 Table 2.2 The Sample Verb and Sample Behavior of Bloom

Original Taxonomy……… 28

Table 2.3. The Cognitive Level of Thinking ……..……… 31 Table 2.4. The Knowledge Dimension………... 32 Table 3.1. Total of Respondents and School Included in the Study ….. 38 Table 3.2. Blue Print of PISA Assessment of Scientific Literac……… 42 Table 3.3. Critical Level of Question………. 50 Table 3.4. The Percentage Category of Scientific Literacy……..……. 51 Table 4.1. Question Tabulation Based on The Level Of Thinking…… 56 Table 4.2 The description of attained each literacy category………... 60 Table 4.3. The Description of Data for Each Aspect Based on School

Level to Whole Sample……… 62

Table 4.4. The Description of Data for Each Aspect Based on Sex

to Whole Sample……… 63

Table 4.5. The Description of Data for Each Aspect Based on School

Level to PISA age-based respondents ……… 64 Table 4.6. The Description of Data for Each Aspect Based on

Sex to Whole Sample……… .. 65 Table 4.7. The comparison of Indonesia correct answer percentage in

PISA and Research……….. 67




Figure 2.1. The component of PISA scientific literacy framework………. 23

Figure 2.2. Cognitive Domain of Bloom original taxonomy and revision 30 Figure 4.1. Cognitive level distribution of the question……….. 55

Figure 4.2. Knowledge Dimension Distribution of the Question………… 57

Figure 4.3. Question Type Distribution………... 59

Figure 4.4. Students’category of Scientific Literacy………... 60

Figure 4.5. Students’ attainment based on school level………... 61

Figure 4.6. The result of Gender factor to science literacy……….. 63

Figure 4.7. The diagram of school effect to science literacy ……...……… 64

Figure 4.8. The result of Gender effect to Science Literacy………...……. 65

Figure 4.9. Question difficulty indices distribution………. 66

Figure 4.10. Physical exercise unit in PISA OECD’s Assessment ………… 80

Figure 4.11. Item of Physical exercise in Ilmu Pengetahuan Alam – Kemendikbud………. 80

Figure 4.12. Question 1 of Physical exercise unit……….. 81

Figure 4.13. The same question found in Kemendikbud science textbook.... 84

Figure 4.14. The complex multiple choice question no. 38 of Major Surgery. 85 Figure 4.15. Question no. 14 of Cloning unit……….……… 85

Figure 4.16. The Genetically Modified Crops unit ………... 90

Figure 4.17. Question 3 from Acid Rain Unit……….... 91

Figure 4.18. The Tabulation of Gases Source fromKemendkbud Book…… 92

Figure 4.19 The question from kemendikbud science textbook which similar To PISA item………. 92

Figure 4.20. The question and matter from ESIS science textbook which Similar to PISA item………. 93

Figure 4.21. The matters from TS which appropriate to answer Acid Rain Item………... 94







Appendix 1. Bloom New Taxonomy Framework and Knowledge

Dimension.. ………. 118

Appendix 2. Question Tabulation Based on New Bloom Taxonomy and Knowledge dimension and Type of Question………... 119

Appendix 3. Comparison of Question Based on Level of cognition, Knowledge Dimension, and Types of Question…………... 122

Appendix 4. Question Collection from the whole Sources……… 123

Appendix 5. Instrument Test for Scientific Literacy………. 153

Appendix 6. Validity and Reliability of Instrument……….. 170

Appendix 7. Reliability Test………. 172

Appendix 8. Difficulty indices of instrument test………. 173

Appendix 9. Discrimination indices……….. 174

Appendix 10. Students’ responses to each item……….. 176

Appendix 11. Normality Test of the Data……… 190

Appendix 12. Homogeneity test of the Data………..………... 191





1.1. Background

The development of knowledge of science which in accordance with the

rapid advancement of technology requires good education system to prepare

peoples to be able adapt to a novel situation, absorb and filter new information,

and solve various issues they face in life with science-related skills (OECD,

2007). It is in line with the core objective of science education which termed as

scientific literacy (Chaisri, 2014; American Association for the Advancement of

Science, 2014). Scientific literacy is defined as the incorporation of scientific idea

and concepts within and across various scientific disciplines, as well as scientific

practices (Shwartz et al, 2006). It relates to the individual competencies of using

scientific knowledge, identifying questions, and drawing evidence-based

conclusions (OECD, 2013; Master, 2005).

Assessment as an important part of evaluation is tightly interconnected

with curriculum and instruction. It assess whether or not the goals of education are

reached after several teaching and learning process (Edutopia, 2008). Many

strategies can be performed to obtain information about the progress and

achievement as a result of education process through the assessment, including;

class-based assessment, school-based assessment, regional, national and

international scale assessment, according to the objectives of the assessment

(Looney, 2011).



Educational organizations have been developed the international-scale

assessments which focus on the achievement and trends of education system

across countries, within country. The results of this international study could

reflect the quality of education in the member and participating countries, enrich

and enlarge the national view in preparing and facilitating the best way about the

weaknesses and strength of a country, finally able to monitoring the improvement

of the country.

Program for International Student Assessment or PISA as an international

large-scale comparative survey are purposes on monitor the trends of students’

literacy and offers insights for education policies of participating countries

(OECD, 2013). This three-year cycle assessment focuses on 15,3 – 16,2 years old

ability to apply knowledge and skills to contextual life (Master, 2005; OECD,

2013). As the compulsory education in entire world is completed in age 15, it

assumed those with age 15 have been mastered the basic skills and sufficient

knowledge to start entering adult life (OECD, 2013). It also reflects how well the

education prepares the youth to challenge, compete, and solve real-world

problems they encounter in future work place with the knowledge they have

(OECD, 2013; Tao, 2008).

Includes in PISA surveys started from 2000 to 2012, Indonesian

achievement in PISA, especially in science always below the average score of

OECD’s countries (500) and took the fifth last place among all participants (Hadi

& Endang, 2009 ; Burckhardt, 2014; OECD, 2003; OECD, 2004; OECD, 2006



place from 41 participants with average score 393 (OECD, 2003). In second

survey, Indonesia obtained the 38th place from 40 participants in 2003 with mean

score 395 (OECD, 2004). Furthermore, the third survey in 2006 where science

became a major domain of assessment, put Indonesia in the rank 53th from 57

participants (average score 395) (OECD, 2006) and in 2009 with the average

score 383 make Indonesia in the rank 62th from 65 participant countries. The last

survey in 2012 put Indonesia in the rank 64th from 65 countries with mean score

382 (OECD, 2013a).

The finding was also informed that most of students’ were reached the

basic level of scientific literacy. The cognitive load by remembering the simple

facts (like name, fact, term, simple formulation) is categorized at this stage.

Moreover, the students at this level only used the common scientific knowledge

and familiar situations to create or evaluate a conclusion (Rustaman, 2010).

The involvement of Indonesia in this international scale assessment was

drawn interest facts about Indonesia achievement. The data which outlined the

low achievement of Indonesian should be used in further analysis to investigate

the possible factors lowering the 15 years old Indonesian students in PISA


There are many factors affected the successful of learning, but several

study reveals that the textbooks is one of factors which determined what students

have learned. The textbook are widely use as the fundamental sources of

information to basic biological concepts, scientific research process, experimental



access (National Research Council, 1997; Haury, 2000; Teixeira et al, 2011).

Moreover, the teachers are actively use textbooks as their main instructional tool

and reference sources (Cobanoglu & Sahin, 2009) and to give assignments as well

(Özay & Hasenekoğlu, 2007).

Governor regulation of Education National Standard No. 19 (chapter 42,

2005) states every educational unit must have the supporting material of learning

to create the well-regulated and continuum learning. Furthermore, the regulation

of Ministry of Education No. 11 (chapter 1, 2005) also notes that textbook play an

important role to increase education quality in primary and secondary school

(Masduki et al, 2013).

The data of the use of science textbooks as basis for instruction is reported

by Martin et al (in Oates, 2014). It compared the used of science textbook in

England, Singapore, and Finland. About 4 % the textbook are used in England, 68

% textbooks are used as basis for instruction in Singapore, and 94% in Finland. It

also reported that the less of textbook used as instructional media might relates to

the lower score attain by England compared to those countries. In addition, the

Singapore and Finland always be the top performer in PISA test (Oates, 2014).

Previous study about the quality of the question embedded in students

textbook reveals that evaluation in junior high school textbook of math have the

applying aspect about 66% - 92%, where the reasoning aspect only 0.39% -

11.63% (Masduki et al, 2013). The low percentage of reasoning aspect makes the

students is not accustomed to solve the challenging problem, creative, analytic and



In carrying out the survey, PISA measured students’ literacy skills through

a set of questions which arranged in booklets of questions. The textbook also have

the subsets of question to evaluate what students learned through a learning

process. Most of schoolteacher usually had the students to complete the exercise

in textbook as the part of assessment after being taught. The quality of the

question within biology exercises mostly posed to the students, especially in

textbook were presumably be one of the factors determining students’ success in

achieving goals in learning biology. It also expected that the format of the

question was liable in determining students’ success in responses a kind of test.

The tendency to answer the question they get used to face with will help

them minimize the time consumption to answering test which include those

common types of questions. It was presumable that the low achievement of

students in PISA survey caused by the limited familiarity of the students to the

type of PISA question or there are the differences of cognitive level and format of

PISA question with question mostly posed by the students, especially in textbook

(Anagnostopoulou et al, 2012).

The compulsory of education in Indonesia was end in the late of junior

high school. The essence of the matter was taught in the level of junior high

school was estimated influence the result of PISA assessment. Nowadays,

Indonesia is facing the changing of curriculum from KTSP curriculum to 2013

curriculum. The shift to new paradigm in 2013 curriculum pays a focus to

empower and acculturate life-long learning and provide pupils with the key



(Kemendikbud, 2014). But the 2013 curriculum is still implemented in some pilot

schools in Indonesia.

The pre-observation about the printed learning material used in teaching

science in Binjai was done in November 2014. It was found that the public junior

high school in Binjai used the various textbooks which released by government

(DEPDIKNAS and KEMENDIKBUD) and private publisher, which utilizing the

both curriculum, 2013 and KTSP curriculum for teaching science.

Since the sampling of respondents is determined based on age at 15,3 -

16,2 years old, the sample in Indonesia with that age are mostly distributed in two

school level namely junior high school students at grade 9 and senior high school

at grade 10. The different level of school was provided the differences in period of

learning science where the senior high school was having more experience with

science compared to junior high school.

Students’ factor such as sex identity was also collected as the crucial

information in PISA survey. The measurement of this factor could give

information about the tendency of how the boys and girls were prepared to

challenging and taking action in future life. Miller et al (2006) examined 79 high

school students and found that the boys were outperformed girls in the subject of

science. The finding were related to sex differences lead most people to believe

that boys are good in mathematics and science related domain, where girls work

best in verbal related subject (Robertson et al, 2003). The result was similar with

Indonesia achievement in PISA survey 2006 where boys outperformed girls in the



paradoxically different with Canada which shows that the girls were score better

than the boys (Huang, 2010). Another study was also performed that sex

differences shows the differences in behaviors, characteristics, and abilities. It

summarized that females have higher verbal ability than males, where males were

excel in visual-spatial and mathematical ability (Maccoby & Jacklin, 1974).

Students’ sex and school level are large enough to be considered affect the

achievement in learning science, so both of these factors also examined in this


1.2. Identification of Problem

According to the background, the identified problems are:

1. Indonesian achievement in PISA, especially in science always below the

average score of OECD’s countries (500) and took the fifth last place among

all participants.

2. The textbook are widely use as the fundamental sources of information to

basic biological concepts, scientific research process, experimental activities

and very often the only source of knowledge to which the students have


3. The quality of the question within biology exercises mostly posed to the

students, especially in textbook were presumably be one of the factors

determining students’ success in achieving goals in learning biology.

4. Textbooks are widely used in biology education. The studies that were

conducted abroad indicate that 90% of the teachers actively use textbooks as



5. It was presumable that the low achievement of students in PISA survey

caused by the limited familiarity of the students to the type of PISA question

or there are the differences of cognitive level and format of PISA question

with question mostly posed by the students, especially in textbook.

6. The more students have familiarity and exercise with the questions resemble

with the PISA assessment there are a tendency to get better score.

7. The students’ sex and school level was also expected influences the

successful of learning science, especially Biology.

1.3. Scope of Study

Based on the background and problem identification have describe, this

research is concern on:

1. The source of PISA assessment was taken from Take the test, Sample

Questions from OECDs PISA Assessment (OECD, 2009: 193-251).

2. The analysis of question within science textbook was limited only for the

question in biology exercises.

3. The analysis of biology question in science textbook follow the same matter

discussed in PISA assessment.

4. The textbook analyzed in this study were limited in the grade 7 for junior

high school.

5. The analysis of PISA items and biology items in science textbook use the

Bloom’s new taxonomy (C1-C6) cognitive level and knowledge dimension,



6. Students’ achievement on PISA (scientific literacy and its’ competencies of

identifying scientific issue, explaining phenomena scientifically, and use

scientific evidence) were measured by administered the students with biology

question from PISA assessment.

7. The role of students’ sex to PISA achievement was analyzed by use the

independent sample t-test by comparing the average score of both sample


8. The role of school level to PISA achievement was analyzed by use the

independent sample t-test by comparing the average score of both sample


9. Sample of research were limited to most frequent science textbooks used in

teaching science and student in age 15, 3 to 16,2 years old which

approximately in grade 9 and 10 in public school in Binjai.

1.4. Research Question

In accordance with the issues that have been stated, then the problem can

be formulated as follow:

1. How are the designs of PISA test item and biology assessment in junior high

school science textbook in Binjai based on new Bloom taxonomy (C1-C6)

and knowledge dimension?

2. How is the design of PISA test item and biology assessment in junior high



3. How are the students’ scientific literacy based on PISA competencies in the

aspect of identifying scientific issues, explaining phenomena scientifically,

and using scientific evidences in Binjai?

4. How are the students’ responses to the questions being tested?

5. Do the sex differences have role to students’ scientific literacy and

competencies in the aspect of identifying scientific issues, explaining

phenomena scientifically, and using scientific evidences in Binjai?

6. Does the school level have role to students’ scientific literacy and

competencies in the aspect of identifying scientific issues, explaining

phenomena scientifically, and using scientific evidences in Binjai?

1.5. Research Objectives This study is aimed to:

1. Reveals the design of PISA test item and biology assessment in junior high

school science textbook in Binjai based on Bloom new taxonomy (C1-C6)

and knowledge dimension.

2. Reveals the design of PISA test item and biology assessment in junior high

school science textbook in Binjai based on the type of question.

3. Reveals students’ scientific literacy based on PISA competencies in the

aspect of identifying scientific issues, explaining phenomena scientifically,

and using scientific evidences in Binjai.



5. Examine the sex differences effect to students’ scientific literacy and

competencies in the aspect of identifying scientific issues, explaining

phenomena scientifically, and using scientific evidences in Binjai.

6. Examine the school level effect to students’ scientific literacy and

competencies in the aspect of identifying scientific issues, explaining

phenomena scientifically, and using scientific evidences in Binjai.

1.6. Research Significances 1.6.1. Theoretical Significance

1. Providing information about the cognitive maps of biology PISA assessment

and biology assessment in students’ science textbook.

2. Encouraging the educators and publisher to create better question which

stimulates students to have higher order thinking skills and able to solve the

problem in real-life situation.

3. As a reference to get information about students scientific literacy in Binjai

which measured based on PISA released item and for other researchers who

want to continue and develop this research.

1.6.2. Practical Significance

1. It can be used as a reference to develop a better quality of question in

science textbook for Junior High School.

2. As a reference to educators to make students become familiar with the

question in real life situation, solving complex problem, and initiates them





5.1. Conclusion

1. There were the differences between the items construction in PISA

assessment and items construction in biology assessment inside the textbook.

The PISA item were mostly distributed in the higher order thinking question

(C4-C6) while the three textbook arranged by the 2006 curriculum were

accumulated into low order thinking question (C1-C3). The assessment inside

the book supplemented by 2013 curriculum delineated the most appropriate

type of assessment to stimulate students higher order thinking skills which in

line with PISA, where the assessment were having the complexity and adapt

to suit the real life situations (more contextual). Moreover, this book also

contains two unit of biology content of PISA assessment.

2. The findings for the type of question being asked revealed that the PISA

assessment mainly composed of four types of question including; simple

multiple choice, complex multiple choice, closed constructed responses, and

open constructed responses. The format of complex multiple choice were not

found inside the three textbook with 2006 curriculum and only 1 item inside

the 2013 curriculum. The limited familiarity of the students to the type of

complex multiple choice was assumed affect the result of the study.

3. Students’ scientific literacy based on PISA assessment in Binjai revealed that

the predominantly samples was categorized low literacy (48.92 ± 13.24). The

PISA competencies are composed of identifying scientific issue, explain



phenomena scientifically, and using scientific evidence. The score attained

for each competency were 46.77 ± 18.77, 49.48 ± 14.19, and 46.30 ± 18. 69

(low category).

4. The students’ responses to each item revealed that there were 12 items

includes into difficult item, 20 questions includes into moderate level

question and 8 items includes into easy item. The difficult items were

appeared might related to the complexity of the question being asked, the

unfamiliar settings and long text provided inside the unit as the illustration of

the context. Furthermore, the complexity of the diagram, chart, tables,

photographs, and the passage of text need students’ good reading skills to

able solving the issues proposed in the unit of assessment. The Indonesian

students reading literacy was also low in PISA assessment were related to the

result of science literacy.

5. Even though the mean score of male respondents were higher than the female,

the Mann-Whitney U test showed that there were no significant differences

between the male and female attainment in scientific literacy and its

competencies in identifying scientific issue, explaining phenomena

scientifically, and using scientific evidence in both age groups.

6. The Mann-Whitney U test showed the significant differences between the

school level where the senior high school outperformed the junior high school

significantly in the aspect of and using scientific evidence (U =69.801.50; P =

0.000, 1st group data; U = 5.405.00; P = 0.043, 2nd group data) in both age



5.2. Implication of Research

The implication of the study are lies in the fact that the books arranged

based on KTSP curriculum still accumulated in the factual knowledge dimension

and assessing students’ lower order thinking skills which might be the one

possible factor in determining students’ success in answering the items. In

addition, the absence of the complex multiple choice item in these three textbook

and students’ limited familiarity with this type of question also estimated affecting

the result of cognitive test. Moreover, the development of the questions in books

which stimulates students higher order thinking skill need to be reconsider.

The effect of international study also drives a changing in curriculum

which shows that Indonesia are in the process of using 2013 curriculum with

emphasizing the scientific learning. The finding of the study concludes that the

book supplemented with 2013 curriculum were the closed one with the PISA

assessment of its format and contextuality. The implementation and the usage of

the books published by Kemendikbud with 2013 curriculum might increase

students’ scientific literacy.

The results of scientific literacy outlines that the predominantly students

were having low level of literacy which measured by PISA questions. The low of

students’ achievement in this case, might related to the construction of the unit

which included the long passages text as the illustration of the context of the unit.

The students’ reading skill and willingness to answer could be viewed as the other



5.3. Suggestions

1. Educators and publisher should be reconsidering the arrangement of

questions which stimulates students’ higher order thinking skills and

provide problem in real-life situation.

2. The used of book by Kemendikbud which supplemented with 2013

curriculum need to be implemented in order to providing the more

contextual teaching and scientific activity.

3. For teachers, it need to make the students be accustomed to solving the

issues related to real-life situations so that the knowledge not only being




American Association of Advancement of Science [AAAS]. (2014). The Nature of Science. Benchmark (Online) Project 2061. Retrieved at

Anagnostopoulou, K., V. Hatznikita, and V. Christidou. (2012). PISA and Biology School Textbok: The role of Visual Material. Social and Behavioral Science 46 (2012) 1839-1845

Anderson, L. W., and D. Krathwohl. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. New York: Longman.

Angelo, T. A., and K. P. Cross. (1993). Classroom assessment techniques: A handbook for college teachers. San Francisco: Jossey-Bass.

Arikunto, S. (2009). Dasar-Dasar Evaluasi Pendidikan. Jakarta: Bumi Aksara

Arikunto, S. (2009). Prosedur Penelitian Suatu Pendekatan Praktek. Jakarta: Rineka Cipta

Becker, B. J., Chang, L., & Michigan, S. U. (1986). Measurement of science achievement and its role in gender differences.

Behringer, F. and M. Coles. (2003). The Role of National Qualification Systems in Promoting Lifelong Learning. OECD Education Working Papers, No.3, OECD Publishing.\

Bielinski, J., & Malison, M. L. (2001). A sex difference by item difficulty interaction in multiple choice mathematics items administered to national probability sample. Journal of Educational Measurement. Spring 2001, Vol. 38, No. 1. Pp. 51-77.



Burckhardt, P. (2014). On the Factors Influencing Performance of Indonesian Students in PISA. Article

Bybee, R., B. McCrae, and R. Laurie. (2009).PISA 2006 : An Assessment of Scientific Literacy. Journal of Research in Science Teaching Vol. 46, No. 8, PP.865-883 (2009).

Chaiseri. (2010). Mathematical and Scientific Literacy Around The World. Journal of Science and Mathematics Education in Southeast Asia 2010, Vol. 33 No. 1, 1-16

Chang, S.N., and M. H. Chiu. (2005). The Development of Authentic Assessment to Investigate ninth Graders Scientific Literacy. International journal of Science and Mathematic Education (2005) 3: 117-140. Taiwan

Cobanoglu, E.O., and B. Sahin. (2009). Underlining the Problems in Biology Textbook for 10th Grades in High School Education Using the Suggestions of Practicing Teachers. (J) Turkish Science EducationVol. 6 Issue 2

Cowell, P. E., Sluming, V. A., Wilkinson, I. D., Cezayirli, E., Romanowski, C. A. J., Webb, J. A., Keller, S. S., Mayes, A., & Roberts, N. (2007). Effects of sex and age on regional prefrontal brain volume in two human cohorts. European Journal of Neuroscience, 25(1), 307-318. doi:10.1111/j.1460-9568.2006.05281.x

Eckes, T., & Trautner, H. M. (Eds.). (2000). The developmental social psychology of gender. NJ: Lawrence Erlbaum Associates, Inc.

Ekohariadi. (2009). Faktor-Faktor yang Mempengaruhi Literasi Sains Siswa Indonesia Berusia 15 Tahun. Jurnal Pendidikan Dasar, 10:28-41

Fensham, Peter J. (2009) Real world contexts in PISA science : implications for context-based science education. Journal of Research in Science Teaching, 46(8), pp. 884-896.



Government of Canada. (2013). The Scientific Literacy in PISA. (online) Retrieved from

Hadi, S. and M. Endang. (2009). Model Trend Prestasi Siswa Berdasarkan Data PISA Tahun 2000, 2003, dan 2006. Laporan Penelitian Pusat Penilaian Pendidikan Badan Penelitian dan Pengembangan Departemen Pendidikan Nasional

Hatzinikita, V., K. Dimopoulos, and V. Christidou,. (2008). PISA test items and school textbooks related to science: A textual comparison. Science Education, 92(4), 664-687

Haury, D.L. (2000). High school biology textbooks do not meet national standards. Eric Document: ED463949.

Hines, M. (2004). Brain gender. New York: Oxford University Press.

Holbrook, J. and M. Rannikmae. (2009). The Meaning of Scientific Literacy. Estonia: (J) IJSE Vol.4 No.3 (2009) 275-288

Huang, C.I. (2010). A Multilevel Analysis of Scientific Literacy: The Effects of Students Sex, Students’ Interest in Learning Science, and School Characteristics. A Thesis. University of Victoria

Huitt, W. (2011). Bloom et al.’s taxonomy of the cognitive domain. Educational Psychology Interactive. Valdosta, GA: Valdosta State University.

Iowa State University. (2012). A Model of Learning Objectives based on A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. Center for Excellence in Learning and Teaching. Iowa State University

Kementrian Pendidikan dan Kebudayaan. (2014). Pembelajaran Saintifik. Kementrian Pendidikan dan Kebudayaan



Lutz, S., & Huitt, W. (2004). Connecting cognitive development and constructivism: Implications from theory for instruction and assessment. Constructivism in the Human Sciences,9(1), 67-90.

Maccoby, E. E., & Jacklin, C. N. (1974). The psychology of sex differences. CA: Stanford University Press.

Manitoba. (2014). Assessment. (online)

Marbach-Ad, G., and P.G. Sokolove. (2000). Good Sciences Begins with Good Questions: Answering the Needs for High-Level Questions in Science. Journal of College Science Teaching Vol. 30 No.3

Masduki, Subandriah, M.R., D. Y. Irawan, and A. Prihantoro. (2013). Cognitive Level of Questions in Middle School Textbook of Math. Yogyakarta: Paper Presented in National seminar of Math and Mathematic Education. (November, 9th 2013)

Masters, G. (2005). International Achievement Studies: Lessons from PISA and TIMMS. Australia: ACER Education Research Development Vol. 13(2005) Article 2

Miller, P. H., Blessing, J. S., & Schwartz, S. (2006). Gender differences in high-school students’ views about science. International Journal of Science Education, 28(4), 363-381. doi:10.1080/09500690500277664

Murphy, P. (1997). Gender differences: Messages for science learning. In K. Harnquist and A. Bergen (Eds.), Growing up with science: Developing early understanding of science. London: Jessica Kingsley.

Murphy, P. (1999). Supporting collaborative learning: A gender dimension. In P. Murphy (Ed.), Learners, learning and assessment. London: Paul Chapman Publishing and Open University.

National Research Council. (1997). National Science Education Standards. Washington DC: National Academy Press



Norris S.P., and L.M. Phillips (2003). How literacy in its fundamental sense is central to scientific literacy. Science Education, 87, 224-240.

Northern Illionis University.(2012). Formative and Sumative Assessment.

Oates, T. 2014. Why Textbook Count. Britain: University of Cambridge

OECD. (2003). Literacy Skills for the World of Tomorrow: Further Result From PISA 2000. Paris: UNESCO Institute for Statistic.

OECD. (2004). Learning for Tomorrow’s World: First Result from PISA (online) available at:

OECD. (2006). Assessing Scientific, Reading, and Mathematical literacy: A Framework for PISA 2006. Paris: OECD

OECD. (2009a). PISA 2009 Assessment Framework-Key Competencies in Reading, Mathematics, and Science. PISA, OECD Publishing.

OECD. (2009b). Take the Test Sample: Sample Question From OECD PISA Assessments. PISA, OECD Publishing

OECD. (2009c). How do Girls and Boys do in Science?, in Highlights from Education at a Glance 2008, OECD Publishing. -2008-37-en

OECD. (2013a), PISA 2012 Results: What Students Know and Can Do – Student Performance in Mathematics, Reading and Science (Volume I), PISA, OECD Publishing.

OECD. (2013b), PISA 2012 Assessment and Analytical Framework: Mathematics, Reading, Science, Problem Solving and Financial Literacy, OECD Publishing.



OECD. 2015. Beyond PISA 2015: a Longer-Term Strategy of PISA. PISA, OECD Publishing

Özay, E. & Hasenekoğlu, İ. (2007). Some Problems in Visual Presentation of

Biology-3 Textbooks. Journal of Turkish Science Education, 4 (1), 80-91.

Rahmiati, E. (2014). Analisis Kemampuan Literasi Sains Siswa Kelas XI SMA Negeri Se-Kota Padang Sidempuan. Thesis. Universitas Negeri Medan

Ramadhan, D., Wasis. (2013). Analisis Perbandingan Level Kognitif dan Keterampilan Proses Sains Dalam Standar Isi (SI), Soal Ujian Nasional

Rustaman, N. Y. (2010). Indonesian Student’ Scientific Literacy. Article

Rushton, J. P., & Ankney, C. D. (1996). Brain size and cognitive ability: Correlations with age, sex, social class, and race. Psychonomic Bulletin & Review, 3(1), 21-36.

Setiawati, D.M., (2013). Analisis Literasi Sains Guru Biologi SMA dan Penerapannya dalam Proses Mengembangkan LKS Inkuiri. Bandung : Universitas Pendidikan Indonesia

Shopia, G. (2013). Profil Capaian Literasi Sains Siswa SMA di Garut Berdasarkan Kerangka PISA (The Program for International Student Assessment) Pada Konten Pengetahuan Biologi. Bandung : Universitas Pendidikan Indonesia

Shortland, M. (1988). Advocating science: Literacy and public understanding. Impact of Science on Society, 38(4), 305-16.



Steinkamp, M. W., & Machr, M. L. (1983). Affect, ability, and science achievement: A quantitative synthesis of correlational research. Review of Educational Research, 53(3), 369-96.

Tjala, A. (2007). Potret Mutu Pendidikan Indonesia Ditinjau Dari Hasil-Hasil Studi Internasional. Artikel. Universitas Negeri Jakarta

Teixeira, T.C., Sigulem, D.M., and Correia, I.C. (2011). Assessment for the Nutritional Issues Contained In High School Biology Textbook. Rev Paul Pediatr 2011;29(4):560-6

Thomson, S., Hillman, K., and Bortoli, L.D. (2013) . A teacher guide to PISA scientific literacy. Australian Council for Educational Research (ACER) Ltd

Turner. R. (2007). PISA- The Program of International Student Assessment – an overview. Article for ACSPRI

Vernon, P. A., Wickett, J. C., Bazana, P. G., & Stelmack, R. M. (2000). The neuropsychology and psychophysiology of human intelligence. In R. J. Sternberg (Ed.), Handbook of intelligence. (pp. 245-264). New York, NY: Cambridge University Press.

Wijaya, A., Heuvel-Panhuizen, M.V.D., Doorman, M., & Robitzsch, A. (2014). Difficulties in Solving Context-Based PISA Mathematic Task: An Analysis of Students’ Errors. Montana : (J) TME Vol. 11 No. 3 p.555

Wiliam. (2013) Assessment: The Bridge between Teaching and Learning. Voices from the Middle, Volume 21 Number 2 National council of teacher of English

Yip, D. Y, M.M. Chiu, and E.S.C Ho. (2004). Hong Kong Students Achievement in OECD-PISA Study: Gender Differences in Science Content, Literacy Skills, and Test Item Format. Taiwan. International Journal of Science and Mathematic Education, 2: 91-106, 2004.