The Level of Thinking Skills in the Reading Section of EFL Textbook in Indonesia

(1)

Vol. 7, No. 1; April 2023 E-ISSN 2579-7484 Pages 117-125

DOI: http://dx.doi.org/10.29408/veles.v7i1.7672

117

The Level of Thinking Skills in the Reading Section of EFL Textbook in Indonesia

*¹Soni Ariawan, ²Muh Azhar Kholidi, ³Mulyono Putra

1Universitas Islam Negeri Mataram, Indonesia

2Universiti Teknology MARA, Malaysia

3University of St Andrews, Scotland

*Correspondence:

soniariawan@uinmataram.ac.id Submission History:

Submitted: January 03, 2023 Revised: April 03, 2023 Accepted: April 05, 2023

This article is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Abstract

The curricular implementations are represented in the textbook. The textbook is used by students with varying levels of thinking skills; however, it is a good opportunity to encourage students' critical thinking skill levels to learn, understand, apply, evaluate, and create a product from their learning. The present study investigates the level of thinking skills employed in the reading parts of EFL textbooks in Indonesia. The Ministry of Education and Culture officially printed the textbook for senior high school students in grade X.

Content analysis was used to explore the level and frequency of thinking skills, focusing on the task and reading questions in the textbook. The data were quantitatively displayed in the form of a table. It was gained through the coding process based on the theoretical framework of Low Order Thinking Skills (LOTS) and High Order Thinking Skills (HOTS) introduced by Bloom. It was found that the textbook mainly included knowledge and comprehension levels of thinking in the reading tasks and questions. It means that LOTS- based tasks and questions are more dominant than HOTS.

Keywords: LOTS, HOTS; Bloom taxonomy; reading tasks; reading questions

INTRODUCTION

High Order Thinking Skills (HOTS) has become famous terminology introduced by the Indonesian government, where the concept was applied in the national exam. The idea attempts to educate students for the 4.0 industrial revolution, in which they are expected to have strong critical and analytical thinking abilities as education's major purpose.

Evidently, the concept is based on the assumption that critical thinking skills help pupils deal with restrictions and successfully solve challenges. (Sutama et al., 2022). It also deals with the level of expected thinking skill where students are not only able to remember but should possess higher thinking skill (Limbach & Waugh, 2010) that covers the concept of

(2)

http://e-journal.hamzanwadi.ac.id/index.php/veles/index Vol. 7, No.1; 2023

118

knowledge, analysis, evaluation, and ability to solve complicated problems through analytical thinking skill (Madhuri et al., 2012; Mahoney & Harris-Reeves, 2019).

Yet, it is argued that the incorporation of HOTS ideas should be mandated not just in high-stakes tests such as national exams but also in textbooks utilized in the teaching and learning process. Therefore, reading is the most likely component of a textbook to improve students' order thinking abilities because they are highly associated (Facione, 1990).

Students' reading comprehension will increase with reflective thinking about their reading (Paul, 2005). Text base model reading activity that aims to understand simple text literally might not call critical thinking skills. Still, situation model reading, where readers should analyze, synthesize and contextualize the content with existing knowledge and situations, may require a high level of thinking skill (Afflerbach et al., 2015).

Extensive textbook analysis and evaluation research has been conducted with a different focus. Some focus on cultural incorporation (Ariawan, 2020; Elmiana, 2019;

Setyono & Widodo, 2019), gender representation (Koster & Litosseliti, 2021; Lee, 2018;

Sovič & Hus, 2015; Widodo & Elyas, 2020; Yasin et al., 2012), ideology, religion, and identity (Behnam & Mozaheb, 2013; Tran-Thanh, 2020; Vu & Febrianti, 2018; Xiong &

Yuan, 2018), type of standard English whether British or American (Lindqvist & Soler, 2022) and the incorporation of moral values (Canh, 2018; Feng, 2019; Puspitasari et al., 2021). Some addressed the idea of promoting critical thinking in listening skills (Irianti et al., 2022), speaking (Viana et al., 2022), and writing skills (Arifin et al., 2020). However, the present study focused on investigating HOTS and LOTS incorporated in the reading section of an EFL textbook.

Moreover, a related study to the present research investigated three EFL textbooks used in university introductory courses and senior high schools (Freahat & Smadi, 2014).

They studied cognitive levels in open-ended questions in reading sections to identify thinking levels. Content analysis was utilized to analyze and monitor the occurrences of the categories by building tools like a checklist based on Bloom's taxonomy. The study concluded that lower-level thinking processes had been dominantly prescribed in the EFL textbook used at university, while EFL textbooks used at senior high school revealed higher-level thinking skills. However, the difference compared to the previous study was in the object of analysis. It involved reading materials, tasks, exercises,s and context by looking at the topic before and after the main reading.

Therefore, the present study aims to investigate the level of thinking skill prescribed in the EFL textbook used by the English teacher at senior high school. Two research questions need to be explored; what are the levels of thinking skills applied in the reading tasks and the reading questions of the EFL textbook? And how are Higher Order Thinking Skills (HOTS) and Low Order Thinking Skills (LOTS) prescribed in the reading part of the EFL textbook? The present study aims to know the level of thinking skills prescribed in the reading section of the EFL textbook published by the Indonesian government. The study's findings will strongly recommend developing High Order Thinking Skills based on compulsory textbooks in the future.

METHOD

The present study investigates an English textbook Buku siswa Bahasa Inggris SMA Kelas 10 (English Student's Book for 10^th grade), published by the Ministry of Education and

(3)

119

Culture, especially in the reading section includes questions, tasks, or exercises as reading passages. The latest revised version of the textbook in the year 2017, prescribed for EFL learning in high school, was used as the primary resource. The textbook discusses 15 chapters that include 17 reading passages. To select the data source, researchers select 56 reading tasks and 142 questions taken from 17 reading passages in the textbook. The data were derived from the reading tasks and questions in the textbook. It started by examining the reading tasks and questions from all the reading parts. The coding process was repeated twice to three times for consistent results.

Bloom's taxonomy was employed as a theoretical framework to determine whether the level of thinking skills belongs to Low Order Thinking Skills (LOTS) and High Order Thinking Skills (HOTS). The category of both thinking skills was based on the cognitive level of the taxonomy, namely knowledge, comprehension, application, analysis, synthesis, and evaluation. As the present study established some categories, it belongs to the content analysis (Ariawan, 2020). Data in the content analysis were expected to be useful evidence as the answer to hypotheses or research questions.

The content analysis employed three standard steps: data reduction, data presentation, and conclusion and verification (Huberman & Miles, 2012). The present study used a priori coding, where the researcher created a checklist that was strongly based on the theories presented in theoretical frameworks. The checklist was further revisited and revised accordingly, involving experts to validate the checklist prior to the research.

Moreover, the result of coding was presented in tables to show the result of quantitative analysis. The table below contains further information on the domain, example, and keyword of the taxonomies (Bloom & Krathwohl, 1956). This is used as an analytical tool to study the reading section of the textbook and determine which degree of thinking skill a certain question or passage belongs to.

Table 1. Cognitive taxonomies

Domain Example and keywords (verb)

Knowledge: Recall data or

information. Examples: List the names of the main characters in the story.

Keywords: States, selects, recognizes, recalls, knows, identifies, matches, describes, names, lists, defines, labels, outlines, recalls, recognizes, reproduces.

Comprehension: Understand the meaning, translation, and interpretation of instructions and problems. State a problem in one's own words.

Examples: What was the main idea of the story?

Keywords: comprehends, converts, distinguishes, estimates, explains, gives an example, interprets, paraphrases, and rewrites.

Application: use a concept in a new situation or unprompted use of an abstraction. Applies what was learned in the classroom to novel situations in the workplace.

Examples: Using what you know about the structure of the stories read in class, write a new story of your own.

Keywords: applies, changes, computes, demonstrates, discovers, manipulates, operates, predicts, prepares, relates, shows, solves, uses.

(4)

120

Analysis: Separates material or concepts into parts so that its organizational structure may be understood. Distinguishes between facts and inferences.

Examples: Break the story into separate parts, describing how they deal with. Keywords: analyzes, breaks down, compares, contrasts, diagrams, deconstructs, differentiates, discriminates, distinguishes, identifies, illustrates, infers, outlines, relates, selects, separates.

Synthesis: Builds a structure or pattern from diverse elements. Put parts together to form a whole, emphasizing creating a new meaning or layout.

Examples: By combining these two stories about whales, what would you predict about the future of the whale population on earth? Keywords:

categorizes, combines, compiles, composes, creates, devises, designs, explains, generates, modifies, organizes, plans, rearranges, reconstructs, relates, reorganizes, revises, rewrites, summarizes, tells, writes.

Evaluation: Make judgments about

the value of ideas or materials. Examples: Is this a well-written story, in your opinion? Why? Keywords: appraises, compares, concludes, contrasts, criticizes, critiques, defends, describes, discriminates, evaluates, justifies, relates, FINDING AND DISCUSSION

The Level of Thinking Skill in the Tasks of Reading Section

The data consists of 56 tasks taken from 17 passages in 15 chapters of the textbook.

The result of task analysis indicates that Low Order Thinking Skill seems more dominant than High Order Thinking Skills. Knowledge is the most frequent task prescribed in the reading section. It is seen in the table below that knowledge reaches 73.21%, comprehension 5.36%, and application 3.57%. Those High Order Thinking Skill domains are not significantly included in the tasks, with only 10.71%, 5.36%, and 1.78% among analysis, synthesis, and evaluation, respectively.

Table 3. Analysis of tasks in the reading section Level of Thinking

Skill Domain Frequency Percentage

Low

Knowledge 41 73.21%

Comprehension 3 5.36%

Application 2 3.57%

High

Analysis 6 10.71%

Synthesis 3 5.36%

Evaluation 1 1.78%

Total 56 100%

The result indicates that tasks in the reading section of the EFL textbook dominantly employ Low Order Thinking Skills in the entire chapter. Most of the instruction aims to ensure students' knowledge or understanding of general information in the passage. It

(5)

121

might happen because some topics in the reading passages deal with places and persons.

Two topics describe a tourist destination that contains a lot of information about them. The instruction in the task mostly recalls the student's memory of the information. The same case is found in chapters 10 and 11, where the textbook's authors describe Habibie and Cut Nyak Dien as Indonesian heroes. The text dominantly prescribes the details of those heroes. The task consistently set the purpose to test the student's memory. Several vocabularies that indicate knowledge are found, such as "define," "name," "list," "pay attention," etc. The sentence of instruction or tasks mostly does not contain Higher Order Thinking Skill statement that aims to let students find by themselves the instruction.

The second most frequently prescribed domain is analysis, at 10.71%, followed by comprehension and synthesis, at 5.36%. However, tasks that aim to test students' highest-order thinking skill is less prescribed in the textbook. These findings follow previous research (Freahat & Smadi, 2014; Ö. Ulum, 2016) that depicted the lack of HOTS skills in the EFL textbooks. In comparison, critical thinking is part of the soft skill students should achieve because the ability to be critical thinker is needed to explore a complex range of problems (Gyenes, 2021). Furthermore, critical thinking is a skill that makes students independent learners (Yulian, 2021).

The Level of Thinking Skill in the Questions of Reading Section

It is also considered similar to the result of questions analysis where Low Order Thinking Skill is still mostly used in the questions of every single part of the reading.

Knowledge and comprehension are the two domains that tend to be used in the questions.

Table 4: Analysis of questions in the reading section Level of Thinking

Skill Domain Frequency Percentage

Low

Knowledge 95 66.90%

Comprehension 35 24.70%

Application 6 4.20%

High

Analysis 6 4.20%

Synthesis 0 0

Evaluation 0 0

Total 142 100%

Similar to the frequency and percentage in the tasks analysis, Low Order Thinking Skill is more frequently included in 142 questions. Most of the reading section provides comprehension questions to recall students' knowledge or memory of the reading text. In chapter 1, for example, the questions solely ask yes or no questions and short answer questions such as "does she have…" and "where does Hanah study?", "What are Hanah's hobbies?" etc. In some chapters, there will always be two comprehension questions that mostly ask detail information. It is seen that the knowledge domain is very dominant in all chapters, with more than 60%. Another significant domain found in the questions is comprehension, with 24.70%. Most questions related to comprehension aim to trigger students to give their explanations, examples, and interpretation of particular information

(6)

122

in the passage. In addition to Low Order Thinking Skill-based questions, the application is not significantly prescribed as it just reaches 4.20%. One of the examples dealing with the application is the question about how the reader applies the value of reading to their social life in chapter 2. Another example is the question about the relationship between one object and another.

On the other hand, the analysis domain reaches 4.20%, the same as the application, which indicates the lack of HOTS inclusion in the questions. The finding indicates that tasks and questions in the reading section mostly prescribe Low Order Thinking Skills in the entire reading section. This finding confirms the previous research that questions in the reading section of EFL textbooks in Indonesia tend to require Low Order Thinking Skills (O.

G. Ulum, 2016; Widiati, 2012). It seems very contradictive with the goals of Indonesian education stated in the curriculum toward students with sufficient critical thinking. To achieve it, a high level of thinking skills proposed by Bloom should be considered and applied in every passage, task, and question. It will benefit students as a brain exercise and a regular process of growing their thinking skills. Out of 5 steps following critical thinking (Paul & Elder, 2012; Fisher, 2001): pre-reading activity, comprehension of the texts and generating a central idea of a paragraph, analyzing the logic of the text, evaluating the logic as well as writing it (Xu, 2011), it seems that the analyzed textbook merely employees two steps only: pre-reading activity and comprehension. It does not describe the process of interpretation, evaluation, and reaction of students. The lack of HOTS in the EFL textbook might decrease the function and benefit of the textbook as guidance toward a learning target. While in a global context, critical thinking is one of the most required skills in the 21^st century, besides communication, collaboration, and creativity (Joynes et al., 2019). The present finding seems pivotal for teachers, textbook writers, and researchers in the future when they have to collaboratively develop HOTS-based textbooks (Aryani & Wahyuni, 2020).

CONCLUSION

The present study aims to find out the inclusion of HOTS and LOTS, especially in the reading section of an EFL textbook. It is found that the textbook applies Low Order Thinking Skill more rather than High Order Thinking Skill. Knowledge and comprehension are the two most frequently inserted in the tasks and questions. It might happen due to the topic presented in the textbook about places and persons that potentially review students' general knowledge. However, High Order Thinking Skill- based tasks and questions are dominantly represented by the inclusion of analysis. It is a good point to trigger students' critical thinking, as mentioned in the curriculum. It is becoming more problematic because of the revision of the Indonesian curriculum due to the big dream to implement a critical thinking approach in the teaching and learning process.

Achieveing learning objectives comprehensively; cannot depend on teachers only. It should be supported with an aligned curriculum. One of the implementations of the curriculum is reflected in the textbook. The textbook is indeed used by all students all over Indonesia that might have various levels of thinking skills. However, it is an excellent opportunity to trigger students, in whatever condition of their critical thinking skill, to learn to know and understand and to apply, evaluate and create such a product from their

(7)

123

learning. The researchers suggest an EFL textbook should accommodate Higher Order Thinking Skill-based tasks and questions rather than Low Order Thinking Skills. The researchers also suggest that further research dealing with LOTS and HOTS analysis in various EFL textbooks is conducted and can be completed by analyzing teachers' and students' perceptions to gain a comprehensive result.

ACKNOWLEDGMENTS

The authors would like to thank SEAMEO QITEP in Language, who funded this research through RESEARCH GRANTS in 2020.

REFERENCES

Afflerbach, P., Cho, B. Y., & Kim, J. Y. (2015). Conceptualizing and Assessing Higher-Order Thinking in Reading. Theory into Practice, 54(3), 203–212.

https://doi.org/10.1080/00405841.2015.1044367

Ariawan, S. (2020). Investigating Cultural Dimensions in EFL Textbook by Using Byram

Checklist. Register Journal, 13(1), 123–152.

https://doi.org/10.18326/rgt.v13i1.123-152

Arifin, S., Ilyas, H. F., & Sukmawidjaya, M. (2020). Using Journal Entries and Assigned Writing to Promote Students' Critical Thinking. VELES: Voices of English Language Education Society, 4(1), 106–117. https://doi.org/10.29408/veles

Aryani, E. J., & Wahyuni, S. (2020). An Analysis of Higher Order Thinking Skills Realization in Reading Comprehension Questions. Language Circle: Journal of Language and Literature, 15(1), 83–89.

Behnam, B., & Mozaheb, M. A. (2013). Identity, Religion and New Definition of Inclusiveness in Iranian High School EFL Textbooks. Procedia - Social and Behavioral Sciences, 70, 1099–1108. https://doi.org/10.1016/j.sbspro.2013.01.164

Bloom, B. S., & Krathwohl, D. R. (1956). Taxonomy of Educational Objectives: The Classification of Educational Goals. In Handbook I: Cognitive Domain. Longmans.

Canh, L. V. (2018). A critical analysis of moral values in Vietnam-produced EFL textbooks for upper secondary schools. In H. Widodo, M. Perfecto, L. Van Canh, & A. Buripakdi (Eds.),. In Situating moral and cultural values in ELT materials: The Southeast Asian context (pp. 111–129). Springer.

Elmiana, D. S. (2019). Pedagogical representation of visual images in EFL textbooks: A multimodal perspective. Pedagogy, Culture & Society, 27(4), 613–628.

https://doi.org/10.1080/14681366.2019.1569550

Facione, P. A. (1990). Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction Executive Summary "The Delphi Report.

The California Academic Press, 423(c), 1–19.

https://doi.org/10.1016/j.tsc.2009.07.002

Feng, W. D. (2019). Ontogenetic analysis of social values in EFL textbooks in Hong Kong. . . Discourse: Studies in the Cultural Politics of Education, 40(4), 458–473.

https://doi.org/10.1080/ 01596306.2017.1356806.

Fisher, A. (2001). Critical Thinking. An Introduction. In Cambridge University Press.

Cambridge University Press. https://doi.org/10.2307/2019787

(8)

124

Freahat, N. M., & Smadi, O. M. (2014). Lower-order and Higher-order Reading Questions in Secondary and University Level EFL Textbooks in Jordan. Theory and Practice in Language Studies, 4(9), 1804–1813. https://doi.org/10.4304/tpls.4.9.1804-1813 Gyenes, A. (2021). Student perceptions of critical thinking in EMI programs at Japanese

universities: A Q-methodology study. Journal of English for Academic Purposes, 54, 101053. https://doi.org/10.1016/j.jeap.2021.101053

Huberman, A., & Miles, M. (2012). Understanding and Validity in Qualitative Research. In The Qualitative Researcher's Companion. Sage Publications.

https://doi.org/10.4135/9781412986274.n2

Irianti, L., Febriani, R. B., & Friatin, L. Y. (2022). Promoting Students' Higher Order Thinking through Flipped Classroom Model in Listening Comprehension Classes. VELES:

Voices of English Language Education Society, 6(1), 201–214.

https://doi.org/10.29408/veles.v6i1.5060

Joynes, C., Rossignoli, S., & Kuofi, E. F. (2019). 21st century skills: Evidence of issue in definition, demand, and delivery for development contexts. Knowledge, evidence, and learning for development contexts (K4D helpdesk report). Institute of Development Studies.

Koster, Dr. D., & Litosseliti, Dr. L. (2021). Multidimensional perspectives on gender in Dutch language education: Textbooks and teacher talk. Linguistics and Education, 64, 100953. https://doi.org/10.1016/j.linged.2021.100953

Lee, J. F. K. (2018). Gender representation in Japanese EFL textbooks – a corpus study.

Gender and Education, 30(3), 379–395.

https://doi.org/10.1080/09540253.2016.1214690

Limbach, B., & Waugh, W. (2010). Developing higher level thinking. Journal of Instructional Pedagogies, 3(1), 1–9.

Lindqvist, N., & Soler, J. (2022). World Englishes in ELT textbooks in Swedish upper- secondary schools. World Englishes. https://doi.org/10.1111/weng.12599

Madhuri, G. V., Kantamreddi, V. S. S. N., & Prakash Goteti, L. N. S. (2012). Promoting higher order thinking skills using inquiry-based learning. European Journal of Engineering Education, 37(2), 117–123. https://doi.org/10.1080/03043797.2012.661701

Mahoney, J. W., & Harris-Reeves, B. (2019). The effects of collaborative testing on higher order thinking: Do the bright get brighter? Active Learning in Higher Education, 20(1), 25–37. https://doi.org/10.1177/1469787417723243

Paul, R. (2005). The state of critical thinking today. New Directions for Community Colleges.

https://doi.org/10.1002/cc.193

Paul, R., & Elder, L. & F. for C. T. (2012). The miniature guide to critical thinking: Concepts and tools. 27th International Conference on Critical Thinking, 2005(130), 27–38.

https://doi.org/10.1002/pfi.4170340606

Puspitasari, D., Widodo, H. P., Widyaningrum, L., Allamnakhrah, A., & Lestariyana, R. P. D.

(2021). How do primary school English textbooks teach moral values? A critical discourse analysis. Studies in Educational Evaluation, 70, 101044.

https://doi.org/10.1016/j.stueduc.2021.101044

Setyono, B., & Widodo, H. P. (2019). The representation of multicultural values in the Indonesian Ministry of Education and Culture-Endorsed EFL textbook: A critical

(9)

125

discourse analysis. Intercultural Education, 30(4), 383–397.

https://doi.org/10.1080/14675986.2019.1548102

Sovič, A., & Hus, V. (2015). Gender Stereotype Analysis of the Textbooks for Young Learners. Procedia - Social and Behavioral Sciences, 186, 495–501.

https://doi.org/10.1016/j.sbspro.2015.04.080

Sutama, S., Fuadi, D., Narimo, S., Hafida, S. H. N., Novitasari, M., Anif, S., Prayitno, H. J., Sunanih, S., & Adnan, M. (2022). Collaborative mathematics learning management:

Critical thinking skills in problem solving. International Journal of Evaluation and Research in Education (IJERE), 11(3), Article 3.

https://doi.org/10.11591/ijere.v11i3.22193

Tran-Thanh, V. (2020). Queer identity inclusion in the EFL classroom: Vietnamese teachers' perspectives. TESOL Journal, 11(3), e00512. https://doi.org/10.1002/tesj.512

Ulum, Ö. (2016). A Descriptive Content Analysis of the Extent of Bloom's Taxonomy in the Reading Comprehension Questions of the Course Book Q: Skills for Success 4 Reading and Writing. The Qualitative Report, 21(9), 1674–1683.

https://doi.org/10.46743/2160-3715/2016.2172

Ulum, O. G. (2016). A descriptive content analysis of the extent of Bloom's taxonomy in the reading comprehension questions of the course book Q: skill for success 4 reading and writing. The Qualitative Report, 21(9), 1674–1683.

Viana, S. A., Suhandra, R. I., & Ariawan, S. (2022). The Analysis of EFL Students' Level of Thinking Skill in Speaking Activities. 1, 103–109. Tadris Bahasa Inggris IAIN Kediri:

https://faktarbiyah.iainkediri.ac.id/2022/08/24/1st-international-conference-on- english-language-teaching/

Vu, T., & Febrianti, Y. (2018). Teachers' Reflections on the Visual Resources in English Textbooks for Vietnamese Lower Secondary Schools. TEFLIN Journal, 29(2), Article 2. https://doi.org/10.15639/teflinjournal.v29i2/266-292

Widiati, A. S. (2012). The Levels of Thinking Skills in The Reading Comprehension Questions Given by the English Teachers of The Senior High Schools. Magister Scientiae, 0(31), 25–33. https://doi.org/10.33508/MGS.V0I31.45

Widodo, H. P., & Elyas, T. (2020). Introduction to Gender in Language Education. Sexuality &

Culture, 24(4), 1019–1027. https://doi.org/10.1007/s12119-020-09753-1

Xiong, T., & Yuan, Z. (2018). "It Was Because I Could Speak English That I Got the Job":

Neoliberal Discourse in a Chinese English Textbook Series. Journal of Language,

Identity & Education, 17(2), 103–117.

https://doi.org/10.1080/15348458.2017.1407655

Xu, J. (2011). The application of critical thinking in teaching English reading. Theory and Practice in Language Studies, 1(2), 136–141. https://doi.org/10.4304/tpls.1.2.136- 141

Yasin, M. S. M., Hamid, B. A., Othman, Z., Bakar, K. A., Hashim, F., & Mohti, A. (2012). A Visual Analysis of a Malaysian English School Textbook: Gender Matters. Procedia - Social

and Behavioral Sciences, 69, 1871–1880.

https://doi.org/10.1016/j.sbspro.2012.12.140

Yulian, R. (2021). The flipped classroom: Improving critical thinking for critical reading of EFL learners in higher education. Studies in English Language and Education, 8(2), Article 2. https://doi.org/10.24815/siele.v8i2.18366