A Preliminary Study on Localising the CEFR Written Production Descriptor to Malaysian Higher Education Context
Mohd Khairul Abu Sufi1*, Faizah Idrus1
1 Kulliyyah of Education, International Islamic University Malaysia, Gombak, Malaysia
*Corresponding Author: [email protected]
Accepted: 15 May 2021 | Published: 1 June 2021
_________________________________________________________________________________________
Abstract: The embrace of CEFR in the Malaysian context was explicated in The Roadmap, a document explaining Malaysia’s latest language in education reform that occurred in 2015.
With the adoption of CEFR into Malaysia’s English language education, all matters related to English language teaching and learning need to be aligned to CEFR, including those in tertiary education. This paper attempts to unravel the thoughts and beliefs of academic at Malaysian higher learning institutions on the need to localise CEFR into Malaysian higher education context. This qualitative pilot study was conducted with three participants to assess the suitability of a 22-questions interview questionnaire, and at the same time build a case on CEFR localisation to the Malaysian context. In modelling and framing the study and interview questions, the Language Management Theory was utilised. An interview was conducted and a rigorous coding exercise was done to disclose the beliefs of Malaysian higher education academics in regards to CEFR and the need to localise the framework. For this study, research only concerted around the written production skills. Findings revealed that higher education academics are concerned with CEFR and suggests that proper measures of adaptations should be taken before CEFR can be rolled out in institutions of higher learning for their teaching, learning and assessment purposes.
Keywords: CEFR, localisation, Malaysia, higher education, written production
_________________________________________________________________________
1. Introduction
The language in education policy in Malaysia has seen numerous changes since the early 1950s and in retrospection, the switches from English to Bahasa Malaysia and vice versa throughout the years were based on either social or political contexts (Pillai & Ong, 2018). Nestled in the Malay Archipelago, this Southeast Asian nation has witnessed how the English language or Bahasa Malaysia (the Malay language) was introduced and reintroduced into its language in education policy affecting all levels of education (Asmah Haji Omar, 2016).
Malaysia is a nation whose economy is booming; thus policymakers have once again realised that there is a need to strengthen the English language curriculum and produce students who are more proficient in the English language. The realisation stemmed from the fact that globalisation is seen as a challenge to the country’s economy and more proficient English language users are required in the workforce (Nor Liza Ali, 2013). Therefore in 2015, the English in education policy was once again reformed with the introduction of a document called The English Language Education Reform in Malaysia: The Roadmap 2015-2025 (also known as The Roadmap). In The Roadmap, the document spells out the various initiatives that will be undertaken in reforming Malaysia’s English language curriculum based on the
stipulations of the Common European Framework of Reference for Languages (CEFR) (The Roadmap, 2015).
Conceived in Europe and introduced in 2001 (later reintroduced in 2018 with an updated version), the CEFR is a language framework that was constructed with the aim to unify all European nations through the elevation of each nation’s language (Council of Europe, 2018).
Although the framework was initially developed for European states, many other nations around the globe has not fallen short of either adopting or adapting the language framework for its own usage (Foley, 2019), including Malaysia. Malaysia’s acceptance of the CEFR is greatly exemplified through the nation’s utilisation of CEFR aligned textbooks and curriculum for the teaching and learning of English language at primary and secondary level education (Nurul Farehah Mohd Uri & Mohd Sallehhuddin Abd Aziz, 2018).
While Malaysia has rolled out the CEFR aligned curriculum in primary and secondary schools, tertiary educational institutions were indifferent of the language framework. This is exemplified by previous research on CEFR in the Malaysian context that is somewhat limited and is mostly confined to either policy papers or perception studies largely at primary and secondary education levels (Fatima Sabbir, 2019; Jerald & Parilah M Shah, 2018; Kok &
Azlina Abdul Aziz, 2019; Mazidah Mohamed & Mohd Sallehhuddin Abd Aziz, 2018; Nurul Farehah Mohd Uri & Mohd Sallehhuddin Abd Aziz, 2018; Nurul Farehah Mohd Uri & Mohd Sallehhudin Abd Aziz, 2020; Savski, 2020). However, embracing CEFR in Malaysian higher education is postulated in The Roadmap and eventually, all higher learning institutions will need to comply to the tenets of the Malaysian English language education reform by 2025 (Zuraidah Mohd Don & Mardziah Hayati Abdullah, 2019).
As stated in the earlier section of this paper, the CEFR was initially devised for use of European nations (Deygers et al., 2018). Consequently, this specifies that CEFR was composed based on the conventions of European languages and because of this, the CEFR carries with it a strong European linguistic trait (Barni, 2015; Barni & Salvati, 2017; McNamara, 2014; Mohd. Khairul Abu Sufi & Mahani Stapa, 2020). While Malaysian higher educational institutions are required to accordingly adopt and adapt CEFR in their language curriculum, once adopted, the language tests will eventually be assessed through the CEFR descriptors. Skills that are usually assessed in language tests usually comprise of reading, writing, listening, and speaking. However, the skill that is of ultimate importance when assessing a student’s language ability is writing, as this particular skill is said to assist in knowledge transmission and also is a communication enabler (North & Piccardo, 2016). Thus, the CEFR written production descriptor will be used to assess tertiary students’ writing in the future.
While earlier research on CEFR in the Malaysian higher education context is limited to its utilisation in policy enactments (Azirah Hashim & Leitner, 2014; Liyana Ahmad Afip et al., 2019), teaching, learning and assessment (Engku Haliza Engku Ibrahim et al., 2017; Erda Wati Bakar, 2020; Izzah Ismail & Rahani Othman, 2020; Mardiana Idris & Abdul Halim Abdul Raof, 2015; Mardiana Idris & Mohamad Hassan Zakaria, 2016; Norsharina Mohd Yusof &
Mohamad Maslawati, 2020; Nurul Farehah Mohd Uri & Mohd Sallehhudin Abd Aziz, 2020;
Nurul Hidayah Razali & Lilisuriani Abdul Latif, 2019; Zarina Mohd Ali et al., 2018), none investigates whether the published CEFR descriptors are suitable for use in assessing writing tasks of Malaysian students in institutions of higher education. Therefore, there is a need to investigate whether the CEFR illustrative descriptor scales require localisation in order to fit local needs. In the attempt to disclose the answer to this question, we conducted a small scale
pilot study on a few academics teaching English language proficiency courses at Malaysian public institutions of higher learning. This pilot study is a small section of a larger research.
2. Literature Review
CEFR
The CEFR was to be published as a policy paper that discussed questions regarding multilingualism, stimulates the use of a mutual metalanguage, sought to improve curricula and facilitates professional mobility within Europe (Little, 2007). However, the language framework now is more known as an educational language policy document in comparison to its sole purpose to unite Europe (Nguyen & Hamid, 2020), especially in the areas of assessment where its descriptor scales have been said to be most influential component (Little, 2006). This is proven by the acceptance of CEFR by various educational institutions around the globe as a tool to assist in areas of teaching, learning, and assessment. In many nations outside the European continent, the CEFR has made its way as a reference benchmark for language in education purposes.
Although CEFR is viewed by many as comprehensive in its nature, the language framework lacks empirical and theoretical foundations (Barni & Salvati, 2017). This is thought to be a flaw of CEFR as its development (especially the illustrative descriptor scales for the many language skills) was not based on any particular linguistic or language theory. In fact, it was reported that the descriptor for ‘Written production’ was emulated from the ‘Spoken production’ descriptor and such compels researchers to question its suitability as a language assessment tool (Weir, 2005b). Other more current research (Harsch et al., 2020; Harsch &
Seyferth, 2020; Zou & Zhang, 2017) suggest that the CEFR requires prior amendment and adaptation so that the language framework suits the needs of a particular nation adopting CEFR.
In defence, the CEFR manual states that the framework is not conclusive in nature and that users are welcome to mend the framework as to fit their context of utilisation as the developers “have NOT set out to tell practitioners what to do, or how to do it” (Council of Europe, 2018, p. 26)
The strong acceptance of CEFR is vividly seen in other Asian nations where countries like Japan and China view the language framework as fundamental to their language reformulation initiatives. To illustrate, both Japan and China emulated the rudiments of CEFR and concocted a version that suitably addresses the language needs of the countries. Dubbed CEFR-J (Japan) and CCFR-E (China), both language frameworks were constructed with the purpose of uplifting and improving the respective nations’ language education. Vietnam is another example where the CEFR is utilised as a foundation for solid English language education reform in the country. The Vietnamese government has spent a great deal of time to adopt and adapt CEFR to local Vietnamese context. Van Huy and Hamid’s (2015) language policy study on CEFR in the Vietnamese context revealed that the utilisation of CEFR as a language framework is seen as the best solution in reforming Vietnam’s English language education.
In the case of Malaysia where CEFR is not so much in its infancy stage anymore, the borrowing of the European language framework was made obvious through a document called The Roadmap published in 2015.
The Roadmap
Development and launch of The Roadmap (The Roadmap, 2015) by the Malaysian Ministry of Education is a stark indication of the nation’s determination in improving its English language
education due to the demands of globalisation. It was based on this realisation that Malaysian policymakers were determined to reform the nation’s English language educational policy at all levels of education. Through The Roadmap, many initiatives on how the English language education in Malaysia can be improved were put into perspectives. What is interesting about The Roadmap is that an entire section of the document was devoted to the introduction of CEFR in the Malaysian context.
In The Roadmap, a plan ranging a duration of 10 years to enhance the teaching and learning of the English language in Malaysia has been laid out. Its goal is to empower Malaysian youths to grow into users of English language that are productive and skilled, thus preparing them for the actual workforce. Additionally, the 10-year plan spans from basic level education up to tertiary level and this ensures that there is continuity of English language input from the learning institutions for the students.
Based on The Roadmap, the decision to adopt the CEFR in the Malaysian English language education reform was to ensure that students achieve the required standards in the English language to function successfully in the work environment. As such, English language proficiency targets were being set and students were required to achieve a particular bar of standard based on the levels of education that they progress through. For example, students of higher educational institutions are required to achieve CEFR B2 (see Appendix 1 for an overview of the written production illustrative scale) upon graduation in order to be considered as possessing the required standard and proficiency in English that is fit for employment.
With the introduction of CEFR into the Malaysian scenario, the nation’s English language in education policy has once again metamorphosised and impacted Malaysia’s English language curriculum, teaching and learning, and assessment matters. The total makeover suggests that CEFR has comfortably burrowed itself into the Malaysian educational policy.
Writing
In tertiary level education, the language skills that defines a student’s language proficiency is writing. According to Grabe and Kaplan (1996), the skill of writing is pertinent in education at the tertiary level because writing is commonly associated with critical analysis and information interpretation, synthesisation, creation, argumentation as well as promoting and presenting research that is valuable to the expansion of knowledge. In regards to the writing skill also, North and Piccardo (2016) postulate that the skill of writing would enable knowledge to be successfully transmitted and that communication between two or more parties be facilitated, thus enabling effective communication.
Another view on writing explains that writing is typically related to an individual’s personal sociocultural context as “what we write, how we write, and who we write to is shaped by social convention and by our history of social interaction” (Hayes, 1996, p. 5). As there is corroboration between an individual’s language proficiency and cultural background, it is therefore justifiable that Malaysian students’ English language proficiency in writing is predisposed by the solid cultural connection that the students possess. In addition, previous research have indicated that second-language writing is typically related and shaped by the linguistic norms of a writer’s first language (Grabe & Kaplan, 1996; Ramiaida Darmi et al., 2018; Wind et al., 2017). As English in considered a second language in Malaysia, the English language linguistic conventions of Malaysian students may differ from students of other nations whose first language are not bound to the linguistic norms of Bahasa Malaysia.
3. Theoretical underpinning
This pilot study was underpinned by the language management theory (LMT) (Jernudd &
Neustupný, 1987). Language management comprises of the activities undertaken by individuals (simple management) or institutions (organised management) that are aimed at managing either language or communication complications (Nekvapil & Sherman, 2015).
Simple management refers to the act of an individual immediately performing the language correction on their own to rectify the deviation he/she has noted in their discourse. Organised language management, on the other hand is more directed and systematic as it involves several levels of an organisation in order for the language problem to be removed. Notably in LMT, it is said that organised management is reliant on simple management for successful language problem removal at the institutional level.
At the theoretical level, LMT is a four-step cyclical process aimed at recognising language problems (notation of deviation), evaluating the extent of the recognised language problem (evaluating), suggesting and proposing appropriate adjustment steps to remedy the language problem (adjusting), and carrying out the proposed steps so that the noted language problem(s) can be eliminated.
Although LMT is presented as an organised cycle comprising of the four steps mentioned, Jernudd and Neustupný explicate that it may actually commence from any stage of the cycle and may also end at any stage of the cycle that the individual or institution deem, has removed the language complication.
4. Methodology
Research design
In conducting this small scale pilot study, a qualitative research designed is employed to attain valuable data that is rich and thick. This will provide immense benefits to the study as information gained is descriptively attained from participants of the pilot study. Since this research is a pilot study in nature, the qualitative design is limited to collecting data for the purpose of establishing a case worthy of further research.
Participants
To acquire data that will address the pilot study’s research question, participants of this study were elected through a convenience-purposive sampling method. Three (3) participants were selected for this research, and they are the researchers’ colleagues, and personal contacts. The selection of participants were also based on the following inclusion criteria: (i) an academic staff teaching English language proficiency courses in a Malaysian public institution of higher learning; (ii) is currently teaching English language proficiency course, specifically writing;
(iii) involved in the assessment of writing tasks in English; and (iv) possess prior knowledge of CEFR. Figure 1 below provides a brief detail of the study’s participants.
Table 1: Participants details
No Pseudonym Gender Workplace Position
1 Arlina F UP Lecturer
2 Shana F UI Lecturer
3 Ashraff M UI Lecturer
Before the interview, we contacted the participants through WhatsApp messaging. The participants are my personal contacts. We enquired whether they would be interested to participate in my pilot study and they both agreed. We also explained that there will be no monetary gain from the participation and both agreed to it also. Since it was a pilot, we also informed all participants that we were unable to provide them with the list of questions as having prior knowledge of the questions before the interview may contaminate the responses.
This means that all participants may have prepared for the pilot session interview and we would not be able to gain authentic responses and reactions to my interview questions. However, because the questions require the participants to be accustomed to the CEFR written production illustrative scale, we did provide them with a soft copy of the descriptor for their reference during the conduct of the pilot study interview.
Research instruments
The research instrument used in this pilot study comprises of a semi-structured interview. The interview questions were constructed by the researchers based on key terms retrieved and found in literature pertaining to CEFR. In addition, important key terms were also acquired from literature that revolved around the assessment of second language writing. The interview questions were then constructed and structured around Bachman and Palmer’s (1996) language knowledge model, the Council of Europe’s CEFR written production descriptor (2018), and the scoring validity model based on Weir’s (2005a) sociocognitive framework.
We initially constructed 22 interview questions. The questions were developed based on variables that we deemed important from our readings. In framing the interview questions, we also structured the development of the questions based on the tenets of LMT. This was achieved by first aligning the extracted variables from literature into the 4 principles of LMT. After all variables were accordingly placed into the appropriate stage of LMT, questions were then constructed observing the respective stage of LMT. We then constructed the questions by utilising the identified variables at the particular stage of LMT ensuring that the questions also concurred with the respective LMT phase.
Study procedures
Construction of interview questions
Interview questions were developed from variables that have been extracted from literature that is pertinent to this study. Explanation of the questions construction can be retrieved from the Methodology section of this article.
Review of interview questions
After the construction of the questions, they were validated by an expert on the subject matter.
At this stage, the questions were approved and required no adjustment nor amendments.
Selection of participants
Participants were selected based on a set of criterion that we have ascertained. These were explained in the Methodology section of the paper. After establishing contact with the interview participants, a link for a digital consent letter was provided just before the participants’
interview session. After obtaining their consent, only then were we able to conduct the online interview.
Interview
Interviews were conducted online through the Google Meet platform. The adoption justification of such method was based on the current Covid-19 pandemic that has limited
physical interactions. Hence, online interviews were conducted. We recorded the interview for the purpose of interview transcription. There were three interview sessions and each lasted around 35-45 minutes. This duration heeded the suggestion of Jacob and Ferguson (2012) where they elucidated that any interview sessions should not exceed a duration of 90 minutes.
During the interview also, we made notations of the interview questions acceptance and indicated which questions required improvement, or were redundant.
Transcription of interview
The interviews were accordingly transcribed based on the Google Meet recordings.
Coding of interview transcriptions
Open coding of the interview transcriptions were done through Atlas.ti. This allowed for a more systematic and structured coding process as data can be easily managed in the same platform. After the process of open coding, we then grouped the open codes based on similar characteristics and produced axial codes of the study. It was through the categorising of axial codes that themes were produced.
Analysis of findings
Findings were generated based on the themes that emerged during the categorising of open, and axial coding process.
5. Results and discussion
Piloting of interview questions
The instrument that was piloted comprised of the interview questions. There were 22 questions in total. All 22 questions were attempted during both interviews and the participants also attempted all questions. As stated in the earlier part of this paper, notations of each interview questions were made during the interview process. Notations included information on whether the interview question was answered, not understood, required amendment, or redundant. The following explicates the findings of the interview questions piloting.
Language of the interview questions
After an analysis of the transcription, it was discovered that the language of the interview questions was technical and complex. Both R1 and R2 commented that some of the interview questions were hard to understand, or were not comprehensible due to its technical nature. The questions were only answered after explaining what the questions entailed.
Technical terms
The CEFR contains many technical terms that may sound foreign to those who are not familiar with the CEFR framework. This was an oversight on my part as we overlooked that laymen may not be familiar with the terms that are used throughout the CEFR framework. For example, the term ‘written production’ was not well understood by the participants until its meaning was explained.
Redundant questions
Another interesting discovery that was revealed during the piloting is that some of the questions constructed were redundant or repetitive. This was either highlighted by the participants or from our investigation of the transcriptions of both interviews.
Context of questions
We also discovered that some questions required prior knowledge before they could be answered. Therefore, there were some instances that the questions were not answered until a context was provided for the participants to understand.
Modification to instrument
Due to the feedback received during the interview, and the analysis done on the transcriptions, we accordingly modified the interview questions so that they are easier to understand. The modification included the following:
a. Rephrasing all questions.
b. Simplifying the interview questions.
c. Reformulating questions as some were double-barrelled.
d. Splitting multiple questions into smaller questions.
e. Rearranging the position of the questions.
f. Changing the term ‘written production’ to ‘writing’.
g. Changing ‘illustrative scale’ to ‘descriptor’.
h. Addition of context before the actual question.
i. Elimination of interview questions no. 13, 15, and 17.
j. Merging of interview question no. 16 with question no. 14
Pilot study
For this pilot study, we analysed the interviews through a thematic analysis and four (4) different themes emerged. The emerging themes are: (i) limited knowledge on CEFR, (ii) CEFR written production descriptor is confusing, (iii) CEFR written production descriptor requires change, and (iv) users are concerned with the CEFR written production descriptor. A brief discussion of each theme will be offered in the subsequent section.
Limited knowledge on CEFR
From the interview, all three participants explained that their knowledge on CEFR is limited to language assessment. Interestingly, all participants associated the CEFR framework with practices of assessment by explaining that there are different levels of language abilities, and each levels specifies a different set of criteria to fulfil. However, they possessed very limited knowledge of its specifications and relevance to non-assessment contexts as much of their knowledge on CEFR was based on the notion that CEFR is merely an assessment tool. For example, Arlina explained that CEFR is an assessment rubric by saying “I know that CEFR is basically a rubric for all languages across the world to use so that we have the same rubric to evaluate the competency of the language, regardless of the language it is.” (Arlina, DU8).
Ashraff expressed his views on CEFR as “the framework on identifying your proficiency level and…. Em… And that’s about it!” (Ashraff, DU10). Additionally, Shana also exemplified her limited knowledge on CEFR when she explained that CEFR is an approach to determine one’s standard of the English language. She explains this by saying “It is a new approach of English, probably. Something to do with the standard of English…?” (Shana, DU12). These findings concur with literature that divulge CEFR as a prevalent language assessment tool (Little, 2006), thus, exacerbate its function as a language framework in the Malaysian higher education context.
CEFR written production descriptor is confusing
Another finding that emerged from the interview was that all participants informed me that the CEFR written production descriptor is confusing. This was made vivid when I open coded the
interview transcription and phrases that repeatedly surfaced from many of the participants when describing CEFR were “confusing” and additionally from one participant, the phrase
“vague”. To illustrate, Ashraff opined that CEFR is confusing and vague because the wordings of the descriptor is ambiguous. He explains it as follows:
This is confusing and we cannot really determine an essay performance based on… “Oh, B1, can write a short, simple essay on the topic of interest…”, “Oh, B2, can write essay, report and develop arguments systematically…” Because, you see, arguments is very subjective you know…
Sometimes, the essay may have an argument but then again the way that its written is quite… I mean, the students may use simple explanations of the arguments. CEFR is vague (Ashraff, DU32)
Likewise for Shana, she expresses her confusion of the CEFR written production descriptor as follows:
I think if you put everything together, the description together in the descriptor, it might be confusing for the examiners because I believe that report is different from essay, a Task 1 essay or a Task 2 essay. It is a different…
what do we call that… layout… the organization probably be also different… the language used would also be different… So, I think it is not really a good idea to, you know… have everything in one descriptor, probably…
(Shana, DU32)
Arlina on the other hand, finds that the CEFR written production descriptor is clear. When we prodded her to explain how she finds it clear, Arlina informed us that only the wordings of the CEFR written production is clear and easy to understand as it has been written in simple language and does not impede the transference of ideas. Although Arlina expresses her satisfaction of the language used in the CEFR written production descriptor, she however articulates that the descriptor lacks an “in-depth” and “comprehensive” explanation of language abilities at the different levels (Arlina, DU20).
Evidently, the revelation of such findings are congruent with past research that have explicated on the vagueness and the non-clarity of the wordings of the CEFR descriptors in general (Green, 2018; Huang et al., 2018; North, 2007).
CEFR written production descriptor requires change
Based on findings, all participants revealed that the CEFR written production descriptor required amendment and adaptation before it could be used effectively by academics in Malaysian higher educational institutions. For instance, Arlina during her interview expressed the inappropriacy of the CEFR written production scale binding ‘report writing’ and ‘essay’
together into one descriptor. She explained that “using the same descriptor to evaluate both types of writing… for me, it would be unfair, and it would be hard for the evaluators to actually evaluate using the same descriptor” (Arlina, DU22).
Another intriguing finding revealed that all participants agreed for the CEFR written production descriptor to be localised to suit context of English used in Malaysian higher educational institutions. Shana explicated that a localised version of the descriptor would
“greatly” assist her with assessing writing papers as allowance will be made for examiners to accept unique linguistic features of standard Malaysian English (Shana, DU137). Additionally, Shana suggested that different CEFR descriptors should be developed for different levels of education. Ashraff also expounded a localised CEFR descriptor would enable him to become a more effective examiner should the CEFR be used to assess writing. Similar to Shana, Arlina explained that the CEFR written production descriptor should be localised as both academics and students at all levels of education have been exposed to the conventions of standard Malaysian English and are not expected to possess native-like proficiency. Arlina also explains that CEFR was conceived based on European languages conventions and it is also due to this that CEFR should conform to criterions of standard Malaysian English.
These views are also in line with past research that indicates the inadequacy of the CEFR framework to address different needs (Nagai et al., 2013; Savski & Savaki, 2019). Another research conducted by Nurul Farehah Mohd Uri and Mohd Salehhuddin Abd Aziz suggest that
“the adoption and implementation of CEFR is incomplete and in fact, invalid without aligning the framework against English syllabus and assessment” (2020, p. 53). Additionally, the CEFR manual also recommends that users appropriately adopt and adapt the CEFR based on their institutional or individual needs.
Users are concerned with the CEFR written production descriptor
The last emerging theme of this study deals with the participants’ concern of the CEFR written production descriptor. Arlina and Ashraff are of the opinion that there is a need for experts to be available before any institutions embrace CEFR. This stem from the fact that both participants explained the need for CEFR training and retraining of examiners so that the descriptors can be used to its maximum potential. Other concerns of the descriptors deals with areas of curriculum where institutions may need to realign the course materials and learning outcomes to fit the specifications of CEFR.
Another noteworthy finding is that all participants are concerned that the published CEFR written production descriptor was fashioned as a holistic descriptor. The concern of assessing written production texts holistically is that examiners may only award grades based on their impression and not delving into other linguistic elements that may enhance the quality of the written production text. This corresponds to previous research that have illuminated holistic scales as being impressionistic and that written productions texts are graded more reasonably when assessors are provided with an assessment rubric that is analytical (Holzknecht et al., 2018).
The emergence of these themes concludes that there is a need to localise the CEFR written production descriptor to the context of Malaysian higher education and thus answers the research question posed in this study.
6. Conclusion
This small-scale pilot study was conducted to assess the suitability of the interview questions, and simultaneously build a case on CEFR localisation grounded on the discoveries of the interview sessions conducted. Preliminary findings of the study suggest that academics of Malaysian higher educational institution are concerned with the introduction of CEFR at
tertiary level. As this study revolved around the CEFR written production descriptor, concerns that surface through the interviews are related to the effects of CEFR to higher education teaching, learning and assessment of students’ writing skills. Additionally, participants of the study confessed to their limited knowledge of the European framework which is a concern that needs to be addressed by Malaysian policymakers should they perceive that the adoption of CEFR would improve the overall English language teaching and learning delivery at all levels of education. Another thought-provoking finding of this pilot study is that all participants indicated that there is a need to adapt and localise the CEFR written production descriptors to the stipulations of standard Malaysian English. Based on the analysis of the interviews, adaptations and localisations should be made to the CEFR written production descriptor so it addresses the unique linguistic traits of standard Malaysian English.
Since this study acts as a pilot for a larger study we intend to carry out in the near future, limitations include the small number of respondents that were involved in the study. Another limitation is that the study was only conducted with academic from two public Malaysian higher learning institutions located in the central region of Malaysia. For the actual study however, we attempt to cover a larger geographical region of the country, subject to the availability of academics that fulfil the participants selection criterion.
In conclusion, based on the findings revealed through this pilot study, it is safe to recommend that there is a need to localise the CEFR written production descriptor to Malaysian higher education context. This will ensure that the descriptor will suit and address the needs of Malaysian higher educational institutions in their assessment practise of written production texts.
References
Asmah Haji Omar. (2016). Positioning languages in the Malaysian education system. In A. H.
Omar (Ed.), Languages in the Malaysian Education System (pp. 1–30). Routledge.
Azirah Hashim, & Leitner, G. (2014). English as a lingua franca in higher education in Malaysia. The Asian Journal of Applied Linguistics, 1(1), 16–27.
http://ssrn.com/abstract=2418116%5Cnhttp://caes.hku.hk/ajal
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford University Press.
https://doi.org/10.2307/328718
Barni, M. (2015). In the name of the CEFR: Individuals and standards. In B. Spolsky, O. Inbar- Lourie, & M. Tannenbaum (Eds.), Challenges of language education and policy. Making space for people (pp. 40–52). Routledge.
Barni, M., & Salvati, L. (2017). The Common European Framework of Reference (CEFR). In E. Shohamy, I. G. Or, & S. May (Eds.), Language Testing and Assessment (pp. 417–426).
Springer International Publishing. https://doi.org/10.1007/978-3-319-02261-1
Council of Europe. (2018). Common European framework of reference for languages:
Learning, teaching, assessment. Companion volume with new descriptors.
Deygers, B., Zeidler, B., Vilcu, D., & Carlsen, C. H. (2018). One Framework to Unite Them All? Use of the CEFR in European University Entrance Policies. Language Assessment Quarterly, 15(1), 3–15. https://doi.org/10.1080/15434303.2016.1261350
Engku Haliza Engku Ibrahim, Isarji Sarudin, Khairiah Othman, Faridah Abdul Malik, & Ainon Jariah Muhamad. (2017). The assessment of writing within the CEFR scale: A Malaysian
context. Advanced Science Letters, 23(5), 4944–4947.
https://doi.org/10.1166/asl.2017.8968
Erda Wati Bakar. (2020). Can-Do descriptors – Realigning English language curriculum at higher education institution to CEFR. International Journal of Modern Languages And Applied Linguistics, 4(2), 84. https://doi.org/10.24191/ijmal.v4i2.7975
Fatima Sabbir. (2019). Perceived view of teachers towards pentaksiran tingkatan tiga (PT3) (Form Three assessment) English language: A case study. Asian Journal of University Education, 15(3), 34–44. https://doi.org/10.24191/ajue.v15i3.7819
Foley, J. A. (2019). Adapting CEFR for English Language Education in ASEAN, Japan and
China. The New English Teacher, 2, 101–117.
http://www.assumptionjournal.au.edu/index.php/newEnglishTeacher/article/view/3879/2 371
Grabe, W., & Kaplan, R. B. (1996). Theory and practice of writing. Longman.
Green, A. (2018). Linking Tests of English for Academic Purposes to the CEFR: The Score User’s Perspective. Language Assessment Quarterly, 15(1), 59–74.
https://doi.org/10.1080/15434303.2017.1350685
Harsch, C., Peña, I. de la C. C., Baffil, T. G., Álvarez, P. C., & Fernández, I. G. (2020).
Interpretation of the CEFR Companion Volume for developing rating scales in Cuban higher education. CEFR, 3.
Harsch, C., & Seyferth, S. (2020). Marrying achievement with proficiency – Developing and validating a local CEFR-based writing checklist. Assessing Writing, 43(January), 100433.
https://doi.org/10.1016/j.asw.2019.100433
Hayes, J. R. (1996). A new framework for understanding cognition and affect in writing. The Science of Writing: Theories, Methods, Individual Differences, and Applications, April, 1–27.
Holzknecht, F., Huhta, A., & Lamprianou, I. (2018). Comparing the outcomes of two different approaches to CEFR-based rating of students’ writing performances across two European
countries. Assessing Writing, 37(April 2017), 57–67.
https://doi.org/10.1016/j.asw.2018.03.009
Huang, L. fen, Kubelec, S., Keng, N., & Hsu, L. hsun. (2018). Evaluating CEFR rater performance through the analysis of spoken learner corpora. Language Testing in Asia, 8(1). https://doi.org/10.1186/s40468-018-0069-0
Izzah Ismail, & Rahani Othman. (2020). A review of literature on the English language entry requirement for international students into postgraduate programs in Universiti Teknologi Malaysia. Journal of Critical Reviews, 7(11), 543–549.
https://doi.org/10.31838/jcr.07.11.98
Jacob, S. A., & Furgerson, S. P. (2012). Writing interview protocols and conducting interviews:
Tips for students new to the field of qualitative research. The Qualitative Report, 17(42), 1–10.
Jerald, G. G., & Parilah M Shah. (2018). The Impact of CEFR-Aligned Curriculum in the Teaching of ESL in Julau District: English Teachers’ Perspectives. International Journal of Innovative Research and Creative Technology, 4(6), 121. www.ijirct.org
Jernudd, B. ., & Neustupný, J. V. (1987). Language planning: For whom? In Proceedings of the International Colloquium on Language Planning (pp. 71–84).
Kok, N. M., & Azlina Abdul Aziz. (2019). English language teachers’ perceptions on the implementation of CEFR-aligned curriculum among primary schools in Malaysia.
Seminar Wacana Pendidikan 2019, February, 212–222.
Little, D. (2006). The Common European Framework of Reference for Languages: Content, purpose, origin, reception and impact. Language Teaching, 39(3), 167–190.
https://doi.org/10.1017/S0261444806003557
Little, D. (2007). The Common European Framework of Reference for Languages:
Perspectives on the making of supranational language education policy. The Modern
Language Journal, 645–655.
Liyana Ahmad Afip, M. Obaidul Hamid, & Renshaw, P. (2019). Common European framework of reference for languages (CEFR): Insights into global policy borrowing in Malaysian higher education. Globalisation, Societies and Education, 0(0), 1–16.
https://doi.org/10.1080/14767724.2019.1578195
Mardiana Idris, & Abdul Halim Abdul Raof. (2015). CEFR rating scale: Scaling its functionality through ESL learners’ self- and peer assessments. Advanced Science Letters.
Mardiana Idris, & Mohamad Hassan Zakaria. (2016). Gauging ESL learners’ CEFR ratings on oral proficiency in rater training. Man in India, 96(6), 1675–1682.
Mazidah Mohamed, & Mohd Sallehhuddin Abd Aziz. (2018). Juxtaposing the primary school assessment concepts and practices in Singapore and Malaysia. International Journal of Engineering & Technology, 7(3.21), 552. https://doi.org/10.14419/ijet.v7i3.21.17232 McNamara, T. (2014). 30 Years on-evolution or revolution? Language Assessment Quarterly,
11(2), 226–232. https://doi.org/10.1080/15434303.2014.895830
Mohd. Khairul Abu Sufi, & Mahani Stapa. (2020). Should the CEFR illustrative scales be localised to Malaysian higher education standards? A conceptual paper. International Journal of Academic Research in Business and Social Sciences, 10(9), 885–897.
https://doi.org/10.6007/IJARBSS/v10-i9/7882
Nagai, N., Ayano, S., Okada, K., & Nakanishi, T. (2013). Adaptation of the CEFR to remedial English language education in Japan. Language Learning in Higher Education, 2(1), 35–
58. https://doi.org/10.1515/cercles-2012-0003
Nekvapil, J., & Sherman, T. (2015). An introduction: Language management theory in language policy and planning. International Journal of the Sociology of Language, 2015(232), 1–12. https://doi.org/10.1515/ijsl-2014-0039
Nguyen, V. H., & Hamid, M. O. (2020). The CEFR as a national language policy in Vietnam:
Insights from a sociogenetic analysis. Journal of Multilingual and Multicultural Development, 0(0), 1–13. https://doi.org/10.1080/01434632.2020.1715416
Nor Liza Ali. (2013). A changing paradigm in language planning: English-medium instruction policy at the tertiary level in Malaysia. Current Issues in Language Planning, 14(1), 73–
92. https://doi.org/10.1080/14664208.2013.775543
Norsharina Mohd Yusof, & Mohamad Maslawati. (2020). Stakeholders’ perceptions and implications of classroom-based reading assessment: A literature review. Creative Education, 11(08), 1324–1335. https://doi.org/10.4236/ce.2020.118097
North, B. (2007). The CEFR: Development, Theoretical and Practical issues. Babylonia, 1, 22–
29. https://doi.org/10.1074/jbc.M409962200
North, B., & Piccardo, E. (2016). Developing illustrative descriptors of aspects of mediation for the Common European Framework of Reference (CEFR). Language Teaching, 49(3), 455–459. https://doi.org/10.1017/S0261444816000100
Nurul Farehah Mohd Uri, & Mohd Salehhuddin Abd Aziz. (2020). The appropriacy and applicability of English assessment against CEFR global scale: Teachers’ judgement. 3L:
The Southeast Asian Journal of English Language Studies, 26(3), 53–65.
Nurul Farehah Mohd Uri, & Mohd Sallehhuddin Abd Aziz. (2018). Implementation of CEFR in Malaysia: Teachers’ awareness and the challenges. 3L: The Southeast Asian Journal of English Language Studies, 24(3), 168–183. https://doi.org/10.17576/3L-2018-2403-13 Nurul Farehah Mohd Uri, & Mohd Sallehhudin Abd Aziz. (2020). Ascertaining the suitability
of writing syllabus specifications to the CEFR: Subject matter experts’ perspectives.
Issues in Language Studies, 9(1), 1–14. https://doi.org/10.33736/ils.2347.2020
Nurul Hidayah Razali, & Lilisuriani Abdul Latif. (2019). CEFR-based English speaking skill self-assessments by Malaysian graduating non-native English speaking students.
Malaysian International Journal of Research in Teacher Education, 2, 82–93.
Pillai, S., & Ong, L. T. (2018). English(es) in Malaysia. Asian Englishes, 20(2), 147–157.
https://doi.org/10.1080/13488678.2018.1459073
Ramiaida Darmi, Fariza Puteh-Behak, Hazlina Abdullah, Ramiza Darmi, & Wahiz Wahi.
(2018). Variations of L1 use in the English language class. GEMA Online Journal of Language Studies, 18(2), 284–311. https://doi.org/10.17576/gema-2018-1802-19
Savski, K. (2020). Local problems and a global solution: Examining the recontextualization of CEFR in Thai and Malaysian language policies. Language Policy.
https://doi.org/10.1007/s10993-019-09539-8
Savski, K., & Savaki, K. (2019). Putting the plurilingual/pluricultural back into CEFR:
Reflecting on policy reform in Thailand and Malaysia. Journal of Asia TEFL, 16(2), 644–
652. https://doi.org/10.18823/asiatefl.2019.16.2.13.644
The Roadmap. (2015). English language education reform in Malaysia: The roadmap 2015- 2025 (Zuraidah Mohd Don (ed.)). Ministry of Education Malaysia.
Van Huy, N., & Hamid, M. O. (2015). Educational policy borrowing in a globalized world: A case study of Common European Framework of Reference for languages in a Vietnamese university. English Teaching: Practice & Critique, 14(1), 60–74.
https://doi.org/10.1108/etpc-02-2015-0014
Weir, C. J. (2005a). Language testing and validation. In Language Testing and Validation.
Palgrave Macmillan. https://doi.org/10.1057/9780230514577
Weir, C. J. (2005b). Limitations of the Common European Framework for developing comparable examinations and tests. Language Testing, 22(3), 281–300.
https://doi.org/10.1191/0265532205lt309oa
Wind, S. A., Stager, C., & Patil, Y. J. (2017). Exploring the relationship between textual characteristics and rating quality in rater-mediated writing assessments: An illustration with L1 and L2 writing assessments. Assessing Writing, 34(April), 1–15.
https://doi.org/10.1016/j.asw.2017.08.003
Zarina Mohd Ali, Fatimah Ali, Noor Raha Mohd Radzuan, Nik Aloesnita Nik Mohd Alwi, Noor Lide Abu Kassim, & Zuraidah Mohd Don. (2018). Contextualising the CEFR: The Universiti Malaysia Pahang English language proficiency writing test. ICERI2018 Proceedings, 1(April 2019), 4892–4902. https://doi.org/10.21125/iceri.2018.2114
Zou, S., & Zhang, W. (2017). Exploring the adaptability of the CEFR in the construction of a writing ability scale for test for English majors. Language Testing in Asia, 7(1).
https://doi.org/10.1186/s40468-017-0050-3
Zuraidah Mohd Don, & Mardziah Hayati Abdullah. (2019, May 22). The reform of English language education in Malaysia. Free Malaysia Today, 1.
https://www.freemalaysiatoday.com/category/opinion/2019/05/22/the-reform-of- english-language-education-in-malaysia/
Appendix 1
CEFR Written Production Illustrative Scale for Written Reports and Essays
(Council of Europe, 2018)
