Development of a Four-Tier Test with a Metacognitive Perspective Approach on the Topic of Newton's Laws

(1)

Tadris: Jurnal Keguruan dan Ilmu Tarbiyah 7 (1): 25-34 (2022) DOI: 10.24042/tadris.v7i1.9615

Development of a Four-Tier Test with a Metacognitive Perspective Approach on the Topic of Newton's Laws

Bambang Subali¹, Niken Tri Widayati¹*, Felia Oktaviana Sekarningtias¹, Wiyanto¹, Ellianawati¹, Siswanto²

1Department ofPhysics Education, Universitas Negeri Semarang, Indonesia

2Department ofScience Education, Universitas Tidar, Indonesia

______________

Article History:

Received: August 1^st, 2021 Revised: September 10^th, 2021 Accepted: May 28^th, 2022 Published: June 29^th, 2022

_________

Keywords:

Four-tier test,

Metacognitive perspective, Misconception,

Newton's law concept

_______________________

*Correspondence Address:

[email protected]

Abstract: This research aimed to develop a learning evaluation instrument on Newton's Law Materials using a four-tier test with a metacognitive perspective to identify students' thinking abilities from metacognition and students' misconceptions. The research employed the R&D (Research and Development) design with a 4-D development model (define, design, develop, and disseminate). The instrument was constructed by developing twelve questions. There were ten valid and reliable questions with good quality parameters (difficulty level and differentiating power). The researchers analyzed the four-tier test instrument with a metacognitive perspective approach using item difficulty level analysis, item discriminatory analysis, test reliability analysis, and expert validity testing. This research involved 30 participants in the small-scale test and 250 participants in the large-scale trial. The test results showed that the ten questions could identify students' thinking abilities from the perspective of metacognition and misconceptions on Newton's Law material. The findings of the large-scale test showed that students' thinking skills were mapped into high, medium, and low categories.

Students' misconceptions were also mapped on each indicator of the questions compiled. Based on the mapping result, the highest students' misconceptions were found in the aspect of the metacognitive experience, with an average of 76 percent of students experiencing misconceptions.

INTRODUCTION

Physics is a discipline that systematically studies important concepts based on careful observations, measurements, and experiments (Ogundeji et al., 2020). Students can build their abilities from low to high levels, emphasizing understanding concepts in physics (Labra et al., 2012).

Human resources in the twenty-first century are emphasized mastering critical thinking and problem-solving skills (Z.

Hidayat et al., 2019; Plessis, 2015;

Rahman, 2019). Physics learning is

intended as a vehicle to grow thinking skills. This thinking ability is important for problem-solving processes in everyday life (Permatasari et al., 2019;

Saputro et al., 2019; Usmeldi, 2016).

Problem-solving is needed in metacognitive abilities so that students can understand the problems being solved and the physics concepts learned to be more meaningful. Metacognition is involved in learning so that students emphasize awareness in their thinking processes (Jaleel & Premachandran, 2016).

(2)

26 | Tadris: Jurnal Keguruan dan Ilmu Tarbiyah 7 (1) : 25-34 (2022)

Metacognition ability is thinking about how to think about oneself (thinking about thinking) or someone's knowledge about the process of thinking (Diandita et al., 2017). Metacognition is also explained as a process in which a person can think about his way of thinking to build strategies to solve problems (Livingston, 2003). This research maximizes students' metacognitive abilities because the instrument developed uses deep thinking and finds problem-solving to answer.

Metacognitive abilities and their relation to the learning process play an important part in the world of education (Zohar & Dori, 2009) as an effort to increase students' awareness of thinking in learning (R. Hidayat et al., 2018). To measure metacognitive ability, the four- tier test format can be used (Gurel et al., 2017). The four-tier test can identify students’ misunderstandings and familiarize them to account for the answers even though they are incorrect (Kaniawati, 2017). Evaluation using the four-tier test is associated with solving students’ problems so that they can find out the process of answering and solving them (Putranta & Supahar, 2019).

The evaluation instrument is developed to determine the metacognitive ability, which refers to a person's understanding of in-depth knowledge (Handel et al., 2013). The four-tier test developed so far only explores students' understanding of a concept. So far, the four-tier test has only been used to diagnose students' misunderstandings, even though the evaluation instrument developed can also be used to determine the students' thinking flow according to the questions asked. (Handel et al., 2013).

Also, based on observations at several high schools in the ex-residence of Pati conducted by researchers by interviewing teachers, no instrument has been found that has been compiled to identify patterns of students' thinking abilities seen from

the metacognitive perspective using the four-tier test.

Therefore, this research developed a four-level test instrument using a metacognitive perspective approach. The instrument developed, apart from measuring misconceptions, is also able to diagnose the level of students' thinking abilities from a metacognitive perspective.

METHOD

The research design used the Research and Development (R&D) method. The R&D method applied the 4- D development model (Thiagarajan et al., 1974) (Mappalotteng et al., 2015). The development flow chart using the 4-D model in this research can be seen in Figure 1. In Figure 1, there are four main stages carried out in this research, namely (1) define stage (definition), (2) design stage (design), (3) develop stage, and (4) disseminate stage.

Based on Figure 1, Phase 1 (define) aims to obtain information related to learning needs using an analysis of the objectives and limitations of the material used in the study. Stage 2 (instrument design) is used to design the four-tier test evaluation to determine the level of students' thinking skills seen from the metacognition perspective and their misconceptions. This stage consists of compiling indicators and question grids, making assessment rubrics, and compiling four-tier test evaluation instruments.

Stage 3 (development) aims to produce an evaluation instrument that has been improved based on the validation results by experts and the results of the analysis of the small-scale trial stage. Stage 4 (disseminate) aims to implement a four- tier test evaluation instrument that has been tested on several state high schools in the ex-residence of Pati. In this fourth stage, the level of students' thinking abilities from a metacognitive perspective and their misconceptions are also mapped.

In this research, students' thinking

(3)

Tadris: Jurnal Keguruan dan Ilmu Tarbiyah 7 (1): 25-34 (2022) | 27

abilities from a metacognitive perspective were categorized into factual, conceptual,

procedural, planning, monitoring, and evaluation processes.

Figure 1. 4-D Development Model

In this research, the product developed was an evaluation instrument, namely a four-tier test to determine the level of thinking ability with a metacognitive perspective and students' misconceptions. The evaluation instrument developed consists of ten items with graded questions in each item and a validation sheet as an assessment sheet for the evaluation instrument.

The researchers analyzed the data based on the validity, reliability, and

identification of the level of thinking ability with metacognitive perspectives and students' misconceptions of Newton's Law material. The small-scale trial stage in the development step was carried out by testing the content validity of the evaluation instrument. Two subject physics teachers and one physics lecturer carried out the expert validation test. The small-scale testing phase was implemented at a high school with 30 students as respondents.

(4)

28 | Tadris: Jurnal Keguruan dan Ilmu Tarbiyah 7 (1) : 25-34 (2022) Table 1. Category Level of Students' Thinking

Ability from a Metacognitive Perspective (Zakiah, 2017)

Percentage Category

70 % < P < 100 % Low 40 % < P < 70 % Moderate

0.0% < P < 40% High

The participants of the large-scale testing came from three public high schools in the ex-residence of Pati. The developed evaluation instrument was distributed to the 250 participants (tenth- grade students). Each school provided two classes to be analyzed to determine students' thinking abilities with a

metacognitive perspective and misconceptions about Newton's law concepts. The students' thinking levels were categorized into three levels, namely low, moderate, and high categories, as presented in Table 1. The percentage value (P) in Table 1 is obtained through equation 1 (Sudijono, 2012). At the same time, the analysis of misconceptions experienced by students is analyzed based on Table 2.

Table 2. Misconception Analysis Rubric (Fariyani et al., 2016)

Level 1 Level 2 Level 3 Level 4 Criteria

Correct Yes Correct Yes Understand the concept

Correct No Correct No

Do not understand the concept

Correct Yes Correct No

Correct No Correct Yes

Correct No Incorrect No

Incorrect No Correct No

Incorrect No Incorrect No

Correct Yes Incorrect No

Incorrect No Correct Yes

Correct No Incorrect Yes

Misconception

Correct Yes Incorrect Yes

Incorrect Yes Correct No

Incorrect Yes Correct Yes

Incorrect Yes Incorrect No

Incorrect No Incorrect Yes

Incorrect Yes Incorrect Yes

RESULT AND DISCUSSION Defining

The four-tier test evaluation instrument was developed to determine how students think in answering the questions asked, even though the students' answers are incorrect (Liu et al., 2011).

The evaluation instrument is to determine the pattern of physics ability with the metacognitive perspective of students in terms of the combination of answers and student interviews. The results are then grouped into high, medium, and low metacognitive abilities. Students' metacognitive abilities are based on references to a deep understanding of knowledge as a problem by the student (Yusnaeni & Corebima, 2017).

Designing

The form of evaluation instruments is made to determine students' thinking abilities with metacognitive perspectives and students' misconceptions. At this stage, indicators and grids of evaluation instruments are produced (see Table 3).

At this stage, as in the initial stage, 12 questions were made according to the designed grid and indicators.

The form of the evaluation instrument developed was also determined at this stage. In this research, the four-tier test format was selected, consisting of four levels of questions asked. The first level refers to items in the form of multiple choices with four distractors and one correct answer

(5)

(McClary & Bretz, 2012). The second level includes student confidence in answering the previous question (Adadan

& Savasci, 2012). The third level includes the reasons for the answer in the first level (Milenkovic et al., 2016). The researchers

prepared four choices of reasons and one open-ended answer choice if there is no match with the answer choices provided.

The fourth level is the students' belief in the reasons at the third level.

Table 3. Evaluation Instrument Grid and Indicators Metacognitive

Aspect

Metacognition

Indicator Question Indicator Question

Number

Metacognitive Knowledge

Factual Choose the mass specification of an object on an inclined plane based on facts about the object's frictional force.

1

Identify the appropriate events based on Newton's Laws I, II, and III principles.

5 Conceptual Formulating the meaning of Newton's Second Law. 3

Analyze the acceleration of an object based on the illustration of the event.

6 Procedural Implement Newton's second law to determine the

mass of an object according to the illustration of the event.

2

Choose a solution to illustrate the incident related to increasing or decreasing the friction force.

4

Metacognitive Experience

Planning Predicting the magnitude of the acceleration based on the diagram of the force acting according to the illustration of events in Newton's second law.

7

Predict the direction and magnitude of the force based on Newton's third law according to the illustration of the incident.

8

Monitoring Assess the magnitude of the force acting following Newton's second law principles.

9 Solve the action-reaction force relationship according to the illustration of events based on Newton's third law.

11

Evaluation Process

Comparing the magnitude of the force with the effect of the frictional force on an object on a flat plane.

10 Defend the principles of applying Newton's third law

in the illustration of events.

12

Developing

At this stage, a small-scale trial was conducted. The small-scale trial began with validity testing by experts. In this research, expert validation was carried out by two senior physics teachers with more than seven years of teaching experience

and one physics lecturer. The expert validation was carried out to assess each item's material, construction, and language aspects. The analysis results of the three expert validations are shown in Table 4.

Table 4. The Result of Expert Validation on the Four-Tier Evaluation Instrument Expert

Validation Code

Percentage (%) Material

Aspect

Construction Aspect

Language

Aspect Average

V-1 69.64 68.05 71.53 69.74 (Quite Feasible)

V-2 100 95.14 90.97 95.37 (Very Feasible)

V-3 99.70 97.22 93.06 96.67 (Very Feasible)

(6)

30 | Tadris: Jurnal Keguruan dan Ilmu Tarbiyah 7 (1) : 25-34 (2022) Table 5. The Level of Difficulty and Differentiating Power of Questions

No Question Difficulty Level Differentiating Power Interpretation

1 0.47 (Medium) 0.49 (High) The question can be used

2 0.87 (Easy) 0.21 (Medium) The question can be used

3 0.20 (Hard) 0.65 (High) The question can be used

4 0.17 (Easy) 0.62 (Height) The question can be used

5 0.50 (Medium) 0.08 (Low) The question cannot be used

6 0.93 (Easy) 0.28 (Medium) The question can be used

9 0.88 (Easy) 0.17 (Low) The question cannot be used

10 0.55 (Medium) 0.44 (High) The question can be used

11 0.30 (Hard) 0.21 (Medium) The question can be used

After expert validation, the instrument reliability test was carried out.

The reliability test was used to determine the level of consistency of the data based on the field and the data obtained. The reliability test was analyzed using the Kuider and Richardson (KR20) reliability test. The result of the instrument reliability test was 0.68 (r table 0.3061), with 30 respondents. These results indicate that the instrument was reliable because the value of rcount was higher than rtable. The low level of instrument reliability was due to sampling factors.

The students were given limited time to complete the test, so many students’

results were not optimal.

Furthermore, the quality parameters of the items are used, including the level of difficulty and the level of discriminating power of the questions (Quaigrain & Arhin, 2017). The results of the interpretation of the level of difficulty and differentiating power are shown in Table 5. The results show that 33.33% of the questions are categorized as easy, 25% are categorized as moderate, and 41.76% are categorized as difficult.

Problems with low discrepancy are not used because the questions cannot distinguish between students who experience misconceptions or not and cannot distinguish thinking abilities from a metacognitive perspective between students.

Disseminating

At this stage, a large-scale test was conducted with 250 students. The large- scale test used ten questions. Based on Table 5, questions number 5 and 9 were not used. The evaluation instrument in the large-scale test was distributed to three public high schools in the Pati ex- residence, specifically to the tenth-grade students. The large-scale test was intended to identify students' thinking abilities with a metacognitive perspective and analyze the misconceptions experienced by students in Newton's Law material. The results showed that students' thinking abilities from a metacognitive perspective were mapped into low, medium, and high levels in each category of thinking ability. The findings can be seen in Table 6.

Table 6. The Percentage of Physics Ability with Metacognitive Perspective on each Indicator based on the Metacognitive Ability Category

The Category of Thinking

Ability

Percentage of Each Indicator of Students' Thinking Ability with Metacognitive Perspective (%)

Factual Conceptual Procedural Planning Monitoring Evaluation Process

High 87.76 53.06 83.67 46.94 40.82 20.41

Moderate 60.00 30.00 45.00 20.00 10.00 10.00

Low 31.03 1.73 1.72 1.72 6.89 0.82

(7)

Tadris: Jurnal Keguruan dan Ilmu Tarbiyah 7 (1): 25-34 (2022) | 31 Table 7. The Percentage of Students who Understand, Do not Understand, and Misconceptions on each Item

Criteria Percentage of Student Achievement in Each Item

1 2 3 4 6 7 8 10 11 12

Understand 31 28 15 36 40 14 14 6 17 6

Do not understand 28 8 7 12 11 11 13 16 14 12

Misconception 41 64 78 52 49 75 73 78 69 82

Based on Table 6, the percentage of students' thinking ability indicators with a metacognitive perspective was high compared to the other categories. The highest percentage in the students' metacognitive abilities category was found in metacognitive knowledge, namely the factual, conceptual, and procedural knowledge indicators. The metacognitive ability for metacognitive strategy indicators has lower results than metacognitive knowledge in all categories of students' metacognitive ability. These results indicated that students with high, medium and low categories tended to have factual knowledge at the level of their thinking ability (Vukić et al., 2020).

Factual knowledge is the ability of concrete thinking and has the lowest level of abstraction compared to other indicators of metacognitive knowledge (Veenman et al., 2004). The lowest percentage was in the evaluation process indicators for the high, medium, and low categories. Students with high categories in their thinking process can evaluate their work correctly to maintain the concepts they understand (Craig et al., 2020).

Besides analyzing the achievement of thinking skills from a metacognitive perspective, the researchers succeeded in mapping students' misconceptions of Newton's law material at this stage. The findings showed that students' misconceptions about Newton's law were identified in each item. These results indicate that students experience misconceptions about each indicator (Table 3). Referring to Table 3, the misconceptions experienced by many students were in the indicator questions number 3, 7, 10, and 12. In this indicator,

more than 75 % of students experienced misconceptions. The high level of student misconceptions in the aspect of the metacognitive experience was due to the instrument that referred to students' strategies to complete their work.

Students had low problem-solving strategies, so it was difficult to answer questions on the aspects of metacognitive experience. Meanwhile, the indicators for questions number 1 and 6 were less than 50 %. The results showed that teachers or instructors could use the evaluation instrument based on the four-tier test as material to explore students' thinking skills from a metacognitive perspective and identify students' misconceptions of Newton's law material.

CONCLUSION

In this research, it can be concluded that of the 12 questions developed, ten questions are valid, reliable, and possess a good parameter quality (in terms of difficulty level and differentiating power).

Based on the large-scale trial, the ten questions can identify students' thinking abilities from the perspective of metacognition and misconceptions on Newton's Law material. The findings of the large-scale trial showed that students' thinking skills were mapped into high, medium, and low categories. Also, students' misconceptions are mapped on each indicator of the questions compiled.

The developed instrument can be used in learning, especially in physics education, for reference materials as a basis for improving the quality of learning physics at the high school level because learning physics requires a pattern of questions

(8)

that require problem-solving and deep thinking.

ACKNOWLEDGMENT

This research received funding support from DIPA Universitas Negeri Semarang, Research Grant Number B/335/UN37/HK/2021, April 12th, 2021.

We would like to appreciate LP2M Unnes for the support and facilities.

REFERENCES

Adadan, E., & Savasci, F. (2012). An analysis of 16–17-year-old students’

understanding of solution chemistry concepts using a two-tier diagnostic instrument. International Journal of Science Education, 34(4), 513–544.

https://doi.org/10.1080/09500693.20 11.636084

Craig, K., Hale, D., Grainger, C., &

Stewart, M. E. (2020). Evaluating metacognitive self-reports:

systematic reviews of the value of self-report in metacognitive research.

Metacognition and Learning, 15(2), 155–213.

https://doi.org/10.1007/s11409-020- 09222-y

Diandita, E. R., Johar, R., & Abidin, T. F.

(2017). Kemampuan komunikasi matematis dan metakognitif siswa SMP pada materi lingkaran berdasarkan gender. Jurnal Pendidikan Matematika, 11(2), 79–

97.

Fariyani, Q., Rusilowati, A., & Sugianto.

(2016). Pengembangan four-tier diagnostic test untuk mengungkap miskonsepsi fisika siswa sma kelas x. Journal of Innovative Science Education, 4(2), 152–162.

Gurel, D. K., Eryilmaz, A., &

McDermott, L. (2017). Development and application of a four-tier test to assess pre-service physics teachers’

misconceptions about geometrical optics. Research in Science &

Technological Education, 35, 1–23.

https://doi.org/10.1080/02635143.20

17.1310094

Handel, M., Artelt, C., & Weinert, S.

(2013). Assessing metacognitive knowledge: Development and evaluation of a test instrument.

Journal of Educational Research Online, 5(2), 162–188.

Hidayat, R., Zulnaidi, H., & Zamri, S. N.

A. S. (2018). Roles of metacognition and achievement goals in mathematical modeling competency:

A structural equation modeling analysis. Plos One, 13(11), 1–11.

https://doi.org/10.1371/journal.pone.

0206211

Hidayat, Z., Ratnawulan, & Gusnedi.

(2019). Analysis of learning media in developing science textbooks with theme energy in life using integrated model for integrated 21st century learning. Journal of Physics:

Conference Series, 1185(1).

https://doi.org/10.1088/1742- 6596/1185/1/012070

Jaleel, S., & Premachandran. (2016). A study on the metacognitive awareness of secondary school students. Universal Journal of Educational Research, 4(1), 165–

172.

https://doi.org/10.13189/ujer.2016.04 0121

Kaniawati, I. (2017). Pengaruh simulasi komputer terhadap peningkatan penguasaan konsep impuls- momentum siswa SMA. Jurnal Pembelajaran Sains, 1(1), 27–34.

Labra, C. B., Gras-Martí, A., &

Torregrosa, J. M. (2012). Effects of a problem-based structure of physics contents on conceptual learning and the ability to solve problems.

International Journal of Science Education, 34(8), 1235–1253.

https://doi.org/10.1080/09500693.20 11.619210

Liu, O. L., Lee, H. S., & Linn, M. C.

(2011). An investigation of explanation multiple-choice items in science assessment. Educational

(9)

Assessment, 16(3), 164–184.

https://doi.org/10.1080/10627197.20 11.611702

Livingston, J. A. (2003). Metacognition : An Overview.

Mappalotteng, A. M., Nur, H., & Kanan, F. (2015). The development of programmable logic controller tutorial in the form of industrial- based learning material in vocational high schools. International Journal of Engineering and Sciece, 5(5), 49–

58.

McClary, L. M., & Bretz, S. L. (2012).

Development and assessment of a diagnostic tool to identify organic chemistry students’ alternative conceptions related to acid strength.

International Journal of Science Education, 34(15), 2317–2341.

https://doi.org/10.1080/09500693.20 12.684433

Milenkovic, D. D., Hrin, T. N., Segendinac, M. D., & Horvat, S.

(2016). Development of a three-tier test as a valid diagnostic tool for identification of misconceptions related to carbohydrates. Journal of Chemical Education, 93(9), 1514–

1520.

https://doi.org/10.1021/acs.jchemed.

6b00261

Ogundeji, O. M., Madu, B. C., Onuya, C.

C., & State, E. (2020). Scientific explanation of phenomenon, imagination and concept formation as correlates of students’

understanding of physics concepts.

Journal of Natural Sciences Research, 10(3), 10–19.

https://doi.org/10.7176/jnsr/11-16-03 Permatasari, A. K., Istiyono, E., &

Kuswanto, H. (2019). Developing assessment instrument to measure physics problem solving skills for mirror topic. International Journal of Educational Research Review, 4(3), 358–366.

https://doi.org/10.24331/ijere.573872 Plessis, A. Du. (2015). Rethinking

traditional science teaching through infusing ICT learning embedded by a

‘learning-as-design’ approach.

Journal of Baltic Science Education, 14(1), Continuous.

Putranta, H., & Supahar. (2019).

Development of physics-tier tests (PysTT) to measure students’

conceptual understanding and creative thinking skills: a qualitative synthesis. Journal for the Education of Gifted Young Scientists, 7(3), 747–775.

https://doi.org/10.17478/jegys.58720 3

Quaigrain, K., & Arhin, A. K. (2017).

Using reliability and item analysis to evaluate a teacher-developed test in educational measurement and evaluation. Cogent Education, 4(1), 1–11.

https://doi.org/10.1080/2331186X.20 17.1301013

Rahman, M. M. (2019). 21st century skill

“problem solving”: Defining the concept. Asian Journal of Interdisciplinary Research, 2(1), 64–

74. https://doi.org/10.34256/ajir1917 Saputro, A. D., Irwanto, Sri Atun, &

Wilujeng, I. (2019). The impact of problem solving instruction on academic achievement and science process skills among prospective elementary teachers. Elementary Education Online, 18(2), 496–507.

https://doi.org/10.17051/ilkonline.20 19.561896

Sudijono, A. (2012). Evaluasi Pendidikan (p. 67). Raja Grafindo Persada.

Thiagarajan, S., Semmel, D. S., &

Semmel, M. I. (1974). Instructional development for training teachers of exceptional children: A sourcebook.

ERIC. https://doi.org/10.1016/0022- 4405(76)90066-2

Usmeldi. (2016). The development of research-based physics learning model with scientific approach to develop students’ scientific processing skill. Jurnal Pendidikan

(10)

IPA Indonesia, 5(1), 134–139.

https://doi.org/10.15294/jpii.v5i1.58 02

Veenman, M. V. J., Wilhelm, P., &

Beishuizen, J. J. (2004). The relation between intellectual and metacognitive skills from a developmental perspective. Learning and Instruction, 14(1), 89–109.

https://doi.org/10.1016/j.learninstruc.

2003.10.004

Vukić, Đ., Martinčić-Ipšić, S., &

Meštrović, A. (2020). Structural analysis of factual, conceptual, procedural, and metacognitive knowledge in a multidimensional knowledge network. Complexity,

2020(2), 1–17.

https://doi.org/10.1155/2020/940716 2

Yusnaeni, A., & Corebima, A. duran.

(2017). Empowering students

’metacognitive skills on sscs learning model integrated with metacognitive strategy. The International Journal of Social Sciences and Humanities Invention, 4(7), 3476–3481.

https://doi.org/10.18535/ijsshi/v4i5.0 3

Zakiah, N. E. (2017). Pembelajaran dengan pendekatan kontekstual berbasis gaya kognitif untuk

meningkatkan kemampuan

metakognitif siswa. Pedagogy:

Jurnal Pendidikan Matematika, 2(2), 11–29.

https://doi.org/10.25157/teorema.v2i 1.704

Zohar, A., & Dori, Y. J. (2009). Higher order thinking skills and low- achieving students: Are they mutually exclusive? The Journal of The Learning Sciences, 12(2), 145–

181.

https://doi.org/10.1207/S15327809JL S1202