CONTENT VALIDITY AND HOTS ELEMENTS OF READING TEST ITEMS DEVELOPED BY MGMP TEACHER IN MALANG
SKRIPSI
BY
MOCHAMMAD GHUZAINU NIFIKY NPM 21821073126
UNIVERSITAS ISLAM MALANG
FACULTY OF TEACHER TRAINING AND EDUCATION ENGSLIH EDUCATION DEPARTMENT
AUGUST 2020
ii
CONTENT VALIDITY AND HOTS ELEMENTS OF READING TEST ITEMS DEVELOPED BY MGMP TEACHER IN MALANG
SKRIPSI
Presented to
Faculty of Teacher Training and Education University of Islam Malang
in partial fulfillment of the requirements for the degree of Sarjana in English Languange Education
BY:
MOCHAMMAD GHUZAINU NIFIKY NPM 218.210.73.126
UNIVERSITY OF ISLAM MALANG
FACULTY OF TEACHER TRAINING AND EDUCATION ENGLISH EDUCATION DEPARTMENT
AUGUST 2020
ABSTRACT
Nifiky, Mochammad Ghuzainu. 2020. Content validity and HOTS elements of Reading test items developed by MGMP teacher in Malang. Skripsi, English Education Department Faculty of Teacher Training and Education University of Islam Malang. Advisor I: Atik Umamah,S.Pd,M.Pd.; Advisor II: Febti Ismiatun,S.Pd.M,Pd
Keywords: Basic competences, test items, HOTS
Assessment is essential to evaluate the student’s competence during teaching and learning process. It aims to measure the ability of students to master of a subject. A assessment involves test items as tool to measure the students’ ability which the reading test items are the common test items used. In constructing a test items, it cannot be separated from basic competence, since it deals with what student’s must learn and master. Thus, creating good test items based on basic competences are crucial, since it relates to the validity of the test. In addition, to compete in 21st century, it is important for students to have critical thinking which involve three cognitive dimensions; analyzing (C4), evaluating (C5) and creating (C6). These there cognitive dimensions could be called as High order thinking skills (HOTS). Therefore, this study was carried out to investigate the use of basic competences on test items based on content validity and the use of HOTS.
The research design used in this study was qualitative approach since it dealt with analysis of the assessments. There were 50 question items of semester assessment and 50 question items of final year assessment on seven grade of Islamic junior high school’s assessments. The data was collected by asking the permission to English teacher of MTs Noor Rachmat Kec. Lawang Kab. Malang East Java since this school is part of MGMP in Malang then analyzed by using thematic method. Meanwhile to find out the content validity of the test, the researcher used the formula developed by Grogery (2015).
The findings showed that the content validity of semester assessment reached 0.94 and 0.96 on final year assessment. It could be concluded that these assessments had good content validity. In addition, the use of HOTS was found on final year assessment which 15 question items were categorized as HOTS items, where creating (C6) was the highest cognitive level. However, it was none of HOTS items on semester assessment. Thus the use of HOTS in all these assessments are important, since the students of Islamic Junior High School should train in critical thinking.
ABSTRACT
Nifiky, Mochammad Ghuzainu. 2020. Content validity and HOTS elements of Reading test items developed by MGMP teacher in Malang. Skripsi, English Education Department Faculty of Teacher Training and Education University of Islam Malang. Advisor I: Atik Umamah,S.Pd,M.Pd.; Advisor II: Febti Ismiatun,S.Pd.M,Pd
Kata kunci: Kompetensi dasar, item tes, HOTS
Penilaian penting dilakukan untuk mengevaluasi kompetensi siswa selama proses belajar mengajar. Hal tersebut bertujuan untuk mengukur kemampuan siswa dalam menguasai suatu mata pelajaran. Asesmen melibatkan butir soal sebagai alat untuk mengukur kemampuan siswa dimana butir soal tes membaca merupakan butir soal yang umum digunakan. Dalam menyusun soal tes tidak lepas dari kompetensi dasar, karena berkaitan dengan apa yang harus dipelajari dan dikuasai oleh siswa. Oleh karena itu, pembuatan butir soal yang baik berdasarkan kompetensi dasar menjadi sangat penting, karena berkaitan dengan validitas soal tersebut.
Selain itu, untuk berkompetisi di abad 21 ini, penting bagi siswa untuk memiliki kemampuan berpikir kritis yang melibatkan tiga dimensi kognitif; menganalisis (C4), mengevaluasi (C5) dan membuat (C6). Dimensi kognitif ini dapat disebut sebagai keterampilan berpikir tingkat tinggi (HOTS). Oleh karena itu, penelitian ini dilakukan untuk mengetahui penggunaan kompetensi dasar pada butir soal berdasarkan validitas isi dan penggunaan HOTS.
Desain penelitian yang digunakan dalam penelitian ini adalah pendekatan kualitatif karena berhubungan dengan analisis penilaian. Terdapat 50 item pertanyaan penilaian semester dan 50 item pertanyaan penilaian terakhir tahun pada kelas tujuh. Pengumpulan data dilakukan dengan meminta izin kepada guru bahasa Inggris di MTs Noor Rachmat Kec. Lawang Kab.
Malang Jawa Timur karena sekolah ini merupakan bagian dari MGMP di Kota Malang
kemudian dianalisis dengan menggunakan metode tematik. Sedangkan untuk mengetahui validitas isi tes menggunakan rumus yang dikembangkan oleh Grogery (2015).
Hasil penelitian menunjukkan validitas isi penilaian semester mencapai 0,94 dan pada penilaian tahun terakhir 0,96. Dapat disimpulkan bahwa penilaian tersebut memiliki validitas isi yang baik. Selain itu, penggunaan HOTS ditemukan pada penilaian tahun terakhir dimana 15 item pertanyaan dikategorikan sebagai item HOTS, dimana kreativitas (C6) merupakan tingkat kognitif tertinggi. Namun tidak ada satupun item HOTS pada penilaian semester. Oleh karena itu, penggunaan HOTS dalam semua penilaian ini menjadi penting, karena siswa SMP Islam harus dilatih berpikir kritis.
1 CHAPTER I
INTRODUCTION
This chapter presents the introduction involving background of study, objective of the study, significance of the study, scope of the study, and definition of the key terms.
1.1 Background of Study
Assessment is essential to evaluate the student’s competence during teaching and learning process. This assessment is conducted in form of examination, and it is used to test the student’s comprehension of what they have learned during the lesson (Ramadhana & Rozimela, 2018). In addition, assessment covers the area of reading, writing, or listening which aims to measure the comprehension of students related to subject (Brown, 2003).& (Harmer, 2007). It purposes to evaluate learning process and give feedback for the teacher, as the result of assessment, it can be used as reference to improve quality of education. In line with this, Permendikbud number 66 the year 2013 states that assessment involves collecting and processing of student’s achievement as well as learning outcome involves Portfolio, daily assessment, Penilaian tengah semester (PTS) or Middle test assessment, Penilaian akhir semester (PAS) or semester assessment, Penilaian akhir tahun (PAT) or final year assessment, Ujian nasional
2
(UN) or national examination and Ujian sekolah (US) or school examination. Those assessments are called as performance assessment, it purposes to give feedback for students in learning process as well as assisting the teacher to build student’s knowledge (Serevina & Muliyati, 2018). Meanwhile, the school’s curriculum provides assessments in written test, listening or reading comprehension test. It is crucial to achieve better quality of schools’ curriculum, therefore the use of assessment is must be done in school to find the strength and weakness of curriculum. Additionally, Permendikbud number 53 year 2015 mentions that the assessment in school can be called as Penilaian Akhir Semester (PAS) or semester assessment, Penilaian Akhir Tahun (PAT) or final year assessment that should be done by students at the end of school year.
Among several types of tests, the reading comprehension test is the most common used and it aims to grasp the concept of the text rather than its meaning (Pourhosein Gilakjani & Sabouri, 2016). Test involves test items that aims to measure the student's ability or mastery a subject (Anderson & Krathwohl, 2001).
The test item plays crucial role in measuring student’s ability, because the result of test aims to evaluate the learning process then create good and effective learning.
Therefore, this result of the test can be reference for teacher to create a good learning’s instruction. The most general used test item is multiple choice test (Kusumawati & Hadi, 2018). Multiple-choice test appears with question and usually more than one answer or choice. The use of multiple choice item occurs on English comprehension test in final test assessment or semester test assessment, it appears in all school levels including Islamic junior high school.
3
Additionally, good test items must be created based on the basic competence of a subject such as attitude, knowledge, and skill that should be possessed by students (Kemendikbud, 2013). This basic competence is the foundation of creating a learning outcome as well as the test items for assessment. The basic competence involves four competencies; spiritual, social, knowledge and skill. The basic competences carries the materials that should be taught and mastered by students during the lesson (Permatasari et al., 2014), the materials-used must be relevant with basic competence as well as in conducting a test. The basic competence is a foundation for creating good test items. The appropriateness between test items and basic competence is important because it relates to the validity of the test. Validity is the appropriateness of the test item with what should be measured (Abdo Rajhy, 2014). Since the use of basic competence on test items are important, therefore this research also aims to know the content validity of the test based on basic competences. Content validity deals with what students have to learn then the materials given must be related to test items (Kristiana, 2014)
Besides, it is important to entangle test items with HOTS (High order thinking skill) items which are relevant with basic competence. HOTS is the ability to be critical thinking through three parts of cognitive dimensions; analyzing, evaluating and creating (Anderson & Krathwohl, 2001). HOTS encourages the students to think critically and solve a problem then implicate all information in making a decision. A revised of cognitive level is classified into knowing (C1) understanding (C2), applying (C3), analyzing (C4), evaluating (C5) and creating (C4) (Anderson & Krathwohl, 2001). Those cognitive levels are categorized as
4
LOTS (Low order thinking) and HOTS. LOTS is knowing (C1), understanding (C2) and applying (C3). Meanwhile HOTS is analyzing (C4), evaluating (C5) and creating (C6). HOTS in the cognitive level demands the ability of students to specify aspects / elements, describe, organize, compare, and find implied meaning as well as encourage the students to construct and find the idea (Setiawati et al., 2018). Therefore the use of HOTS in test items demands the ability of students to use reasoning and logic to make decisions (evaluation), predict and reflecting, as well as the ability to devise new strategies to solve non-routine contextual issues.
A study of HOTS indicates that the significant of HOTS gives effective outcome in the learning process as well as promotes the students to think critically and become creative in solving the problem as a matter of fulfilling the Indonesia’s education needs (Yen & Halili, 2015). Therefore, the use of HOTS items immensely has an impact on students, it encourages student to be high order thinker to create the ability of critical thinking.
Meanwhile, the use of HOTS items through an English reading comprehension test can help the student to open their knowledge to new global topics (Febriana et al., 2019). These HOTS items can assist the students to be problem solver and think critically. These items are carried out reading comprehension test, it is believed that by using HOTS items on reading test can create high order thinking skill and problem solver, moreover it can motivate students to have wide thinking (Valdev Singh et al., 2019). The use of HOTS indicates that students are ready to face any challenges and solve it based on knowledge.
5
The current study of related to content validity presented by (Sa’adah, 2017) and (Nuryanto, 2018) indicated that Mid-test of English in MTs MATHALIBUL HUDA academic year 2016/2017 and English reading test of English national standard school final examination (NSFEE) were good in content validity, since it reached 88 % or 22 items that matched with basic competence of mid-test and English reading test of English national standard school final examination (NSFEE) indicated that this test was good, since the test match to the materials in syllabus reached 90 %. It means that both two tests were good enough based on content validity, however there were bad discrimination power since it reached 20 items were insufficient then need to be revised on Mid-test of English in MTs MATHALIBUL HUDA. In addition, another study of HOTS items on English reading test presented by (Putra & Abdullah, 2019) and (Ramadhana & Rozimela, 2018) indicated that there were less than 27 percent of HOTS question items in national examination of 2013 until 2018, In addition there were only 33 percent of HOTS items on Middle test (PTS) and semester assessment (PAS) in Senior High school of Padang city. These HOTS items were qualified as analyzing (C4), evaluate (C5) and create (C6). It assumed that only a few numbers of test items provide HOTS. As we know the importance of HOTS can create the student to think critically to achieve the triumph of cognitive skill (Abdo Rajhy, 2014). Moreover, based on the study above mentioned that the quality of HOTS items on middle test items and semester test item were not enough to reflect good test items which provide HOTS.
6
Besides, English test items always develop based on the curriculum’s needs.
The previous study indicates that test items of English reading comprehension test has used HOTS to measure the student’s ability in thinking critically. However, the previous study of HOTS above presents a HOTS analysis without finding the use of content validity based on basic competence. The previous studies analyze the use of HOTS items on reading test in the level of High school. However, The HOTS items of English reading comprehension test at Islamic junior high school (MTs) have never been done yet. As the implementation of Keputusan Direktur Jendral Pendidikan Islam or The Decision of the director general Islamic education number 5163 year 2018 mention that the HOTS must be implemented in Islamic junior high school as a prove of 2013 curriculum needs. These HOTS items are developed by Musyawarah guru mata pelajaran or English teacher organization in each region.
In addition the use of HOTS item in Islamic school is not implemented in all assessment subjects, but the common HOTS items present in Islamic subject assessment (Ahmad & Sukiman, 2019). Therefore, the researcher conducts a study to discover what extent do the test items in the reading comprehension test match with basic competence and the quality of HOTS in English reading comprehension test of Islamic Junior High School.
1.2 Research question
The research problem of this study is stated as follow;
7
1. To what extent do the test items in the reading comprehension test of Islamic Junior High School match the basic competence?
2. How does the use of HOTS questions of reading comprehension test in Islamic junior high school (MTs)?
1.3 Objective of the study
The objectives of this study are:
1. To know the extent to which test items in the reading comprehension test of Islamic junior high school match the basic competence
2. To know the use of HOTS questions of reading comprehension test in Islamic junior high school (MTs)
1.4 Significances of the study The study is have impact for:
1. Teacher
This study can be beneficial for teacher to be their references, knowledge and evaluate in making HOTS questions match with basic competence
2. School
8
This study can be reference for the school to evaluate the use of HOTS items in English reading comprehension test questions match with basic competence.
3. Director General Islamic Education
This study can have impact for their reference in constructing a rule related to the implementation of HOTS items match with basic competence in school.
1.5 Scope and Limitation of the Study
This study focuses on the use of HOTS questions and English basic competence developed by the MGMP teacher of Islamic junior high school in Kabupaten Malang academic year 2018/2019. This study investigates the HOTS questions and content validity on PAS (semester assessment) and PAT (final year assessment) of English test items. The limitation of this study is the test item used only in the English reading comprehension test on PAS (semester assessment) and PAT (final year assessment), in addition the assessments are not tested to students to find out the use of logical validity.
1.6 Definition of Key Terms
9
1. PAS (Penilaian akhir semester) is Semester assessment, it is conducted to measure the student's ability of a subject during a half academic year in the odd semester
2. PAT (Penilaian akhir tahun) is final year assessment, is conducted to measure the student's ability of a subject during another half-semester of the academic year in even semester
3. HOTS items is the instrument of measuring high order thinking skill in reading comprehension test item. At this point, HOTS items measure the ability of analyzing, evaluating and creating. In addition, HOTS items involve the process of implementing the information, discover the information, use the information for solving a problem and critically examine ides as well as the information
4. Reading test is an assessment to measure the students’ comprehension of English text, it is provided in form of multiple-choice test. This reading test involves seven grade English comprehension test on semester assessment and final year assessment.
10
48 CHAPTER V
CONCLUSIONS AND SUGGESTION
This chapter deals with conclusions and suggestion. They will be presented as follow:
5.1 Conclusions
After conducting the data collection then discussed it on previous chapter, it can be concluded that there were 0.94 and 0.96 question items of English reading test on semester assessment and final year assessment were match with basic competence. There were more than 0.80 of the test items appropriate to basic competence of 2013 English syllabus. However, it can be concluded that the use of basic competence on the test items cover the content validity. Moreover, the use of HOTS items is also found on the tests. There are 15 question items cover high order thinking skills out of 100 question items, both on semester assessment and final year asessment. At these assessments involved the use of cognitive levels on high order thinking skill items, they were evaluating (C4) and creating (C6). Since the portion of HOTS items only cover less than 50 percent of questions in total, it indicated that the portion of HOTS items were deficient, since at their age, students must be train and familiar with HOTS in order to achieve the ability of critical thinking. Thus, the number of HOTS items in these assessments
49
were not good since it only limited number of HOTS items, it reached 15 question items covered HOTS out of 100 question items analyzed. It can be concluded that all question items were good in measuring the student’s comprehension of English based on content validity. Lastly, that the use of HOTS items were good enough to measure high order thinking skill which aims to make them think critically and creative to compete in globalization.
5.2 Suggestion
Finally, at the end of this study the researcher brings the suggestions for;
among the teacher, curriculum or stakeholder (Director General Islamic Education) takes responsibility in constructing the students’ critical thinking. The test items is designed to test the student in higher order thinking skill then the classroom atmosphere should implement HOTS itself in order to enhance these skills. In other words, the teacher’s training or seminar related to the implementation of HOTS is important. Finally, the portion of HOTS items on English reading comprehension test should improve to emphasize the students’ higher thinking skills. Regarding to the results of this study, since there are still limited number of the research HOTS along with basic competence, it is important to conduct the HOTS study which is not only to know the use of HOTS then match it with basic competence on one year academic year but also two academic years to know the improvement of HOTS used. Besides, the use of HOTS items are also found in semester assessment and final year assessment, it is also found on other assessment such as middle test assessment or school assessment, thus the future study need to examine the use of
50
HOTS in all kind of assessments. In addition, since the limited study of content validity in this study, thus it is important to conduct the study to measure the validity of logical, since it is used to measure to what extend do the test items really represent what should be measured.
51
REFERENCES
Abdo Rajhy, H. A. (2014). Five Characteristics of a good language test. II, 9.
Ahmad, I. F., & Sukiman, S. (2019). Analisis Higher Order Thinking Skills (Hots) Pada Soal Ujian Akhir Siswa Kelas 6 Kmi Dalam Kelompok Mata Pelajaran Dirosah Islamiyah Di Pondok Modern Tazakka Batang. Jurnal Pendidikan Agama Islam, 16(2), 137–164. https://doi.org/10.14421/jpai.2019.162-02 Anderson, L. W., & Krathwohl, D. R. (2001). A Taxonomy for Learning, Teaching
and Assessing (Abridged Edition). Addison Wesley Longman,Inc.
Assessing (Abridged Edition). Addison Wesley Longman,Inc.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative
Research in Psychology, 3(2), 77–101.
https://doi.org/10.1191/1478088706qp063oa
Brookhart, S. M. (2010). How To Assess Higher order thinking skill in your classroom.
Brown, H. D. (2003). Language Assessment Principles and Classroom practices (First). Pearson Education ESL.
California Academic Press. (2016). California Critical thinking skill test. California Academic Press.
Dunn, K. E., & Mulvenon, S. W. (2009). A Critical Review of Research on Formative Assessments: The Limited Scientific Evidence of the Impact of Formative Assessments in Education. 12.
52
Edward, N. (1982). Constructing achievement tets (Third Edition). Prentice- Hall,Inc.
Fachrurrazy. (2014). Teaching english as a foreign language for teacher in Indonesia. State University of Malang Press.
Febriana, Usman, B., & Muslem, A. (2019). Analysis of reading comprehension question using revised bloom’s taxonomy on high order thinking skill (HOTS). English Education Journal, 10(1), 15.
Gregory, R. J. (2015). Psychological Testing (Seventh edition). Pearson Education Limited 2015.
Gronlund, N. E. (1982). Constructing achievement tests (Third). Prentice-Hall,Inc.
Harmer, J. (2007). The practice of english language teaching (fourth edition).
Perason Education Limited.
Kemendikbud. (2013). Kompetensi dasar dan kompetensi inti. Kementerian Pendidikan dan Kebudayaan.
Kristiana. (2014). An Analysis On The Content Validity Of Summative Test For The Second Grade Students Of Junior High School. Syarif Hidayatullah State Islamic University.
Kusuma, M. D., Rosidin, U., Abdurrahman, A., & Suyatna, A. (2017). The Development of Higher Order Thinking Skill (Hots) Instrument Assessment In Physics Study. IOSR Journal of Research & Method in Education (IOSRJRME), 07(01), 26–32. https://doi.org/10.9790/7388-0701052632 Kusumawati, M., & Hadi, S. (2018). An analysis of multiple choice questions
(MCQs): Item and test statistics from mathematics assessments in senior
53
high school. Research and Evaluation in Education, 4(1), 70–78.
https://doi.org/10.21831/reid.v4i1.20202
Majid, A. (2014). Stategi pembelajaran. PT Remaja Rosdakarya.
Medriati, R., Irawati, S., & Ekaputri, R. Z. (2018). Mapping the standard competencies, basic competencies, and the indicators (SKKDI) of natural science course of middle school students in curriculum 2013. Journal of Physics: Conference Series, 1116, 032019. https://doi.org/10.1088/1742- 6596/1116/3/032019
Nuryanto, M. (2018). The Validity Analysis Of Reading Test Items On National Standard School Final Examination For 12th Grade Of Man 1 Semarang.
Universitas Muhammadiyah Surakarta.
Permatasari, I., Agung, L., & Bachri, S. (2014). Implementasi Kompetensi Inti Kurikulum 2013 Dalam Pembelajaran Sejarah (Studi Kasus Di Sma Mta Surakarta). Jurnal UNY, 4.
Pourhosein Gilakjani, A., & Sabouri, N. B. (2016). How Can Students Improve Their Reading Comprehension Skill? Journal of Studies in Education, 6(2), 229. https://doi.org/10.5296/jse.v6i2.9201
Putra, T. K., & Abdullah, D. F. (2019). Higher-Order Thinking Skill (Hots) Questions In English National Examination In Indonesia. Jurnal Bahasa
Lingua Scientia, 11(1), 145–160.
Https://Doi.Org/10.21274/Ls.2019.11.1.145-160
54
Ramadhana, N. A., & Rozimela, Y. (2018). High Order Thinking Skills-Based Questions In The Test Items Developed By Senior High School English Teachers Of Padang. 7(4), 12.
Sa’adah, N. (2017). The Analysis Of English Mid-Term Test Items Based On The Criteria Of A Good Test At The First Semester Of The Eighth Grade Students Of Mts. Mathalibul Huda Mlonggo In The Academic Year Of 2016/2017. 4(1), 14.
Serevina, V., & Muliyati, D. (2018). Development of student performance assessment based on scientific approach for a basic physics practicum in simple harmonic motion materials. Journal of Physics: Conference Series, 1013, 012051. https://doi.org/10.1088/1742-6596/1013/1/012051
Setiawati, W., Asmira, O., Ariyana, Y., Bestary, R., & Pudjiastuti, A. (2018). Buku penilaian berbasis high order thinking skill. Kementerian Pendidikan dan Kebudayaan.
Trilling, B., & Fadel, C. F. (2009). 21st Century Skills: Learning for Life in Our Times. Jossey Bass.
Valdev Singh, R. K., Shaari, A. H., & English Language Studies Programme, Center for Literacy and Sociocultural Transformation. (2019). The analysis of Higher-Order Thinking skills in English reading comprehension tests in Malaysia. Malaysian Journal of Society and Space, 15(01).
https://doi.org/10.17576/geo-2019-1501-02
55
Vianti, S. L. (2011). Kesesuaian Antara Pengembangan Indikator Dan Kompetensi Dasar Dalam Silabus Ktsp Aspek Membaca Di Smp Negeri 3 Batang Tahun Ajaran 2010/2011. Universitas Negeri Semarang.
Widana, I. W. (2017). Higher Order Thinking Skills Assessment (Hots). Jisae:
Journal Of Indonesian Student Assesment And Evaluation, 3(1), 32–44.
https://doi.org/10.21009/JISAE.031.04
Yen, T. S., & Halili, S. H. (2015). Effective Teaching Of Higher-Order Thinking (Hot) In Education. 3(2), 7.