Validity Test Using Principal Component Analysis (PCA) Involving Research Instruments on Learner-Centeredness,
Knowledge-Centeredness, and Assessment-Centeredness
Nyak Mutia Ismail1*& Marisa Yoestara2
1,2Universitas Serambi Mekkah, Banda Aceh, Aceh, Indonesia
*Corresponding Author: [email protected]
Abstract
Instrument is an important element in research, including in educational research. In measuring learning environment formed in a teaching process, discrete variables need to be involved. Thence, understanding that teaching-learning process has varied pedagogical outcome(s), teachers and instructors can focus on their teaching orientation—learner- centeredness, knowledge-centeredness, or assessment-centeredness—and can avoid zoning- out of context during teaching with these factorial discrete variables. There are 15 factors determining the centeredness propensity in a research instrument developed based on Bransfrod et al. (2001). This study aims to simplify and reduce these factors to obtain dominant factors that influence the centeredness in a learning environment by using Principal Component Analysis (PCA). The study was conducted by collecting data through questionnaires to 10 respondents. Finally, 5 major factors were obtained. The main factor has an Eigenvalue of 3.95 with a variance of 26.38%. The second factor has an Eigenvalue of 3.51 with a variance of 23.43%. The third factor has an Eigenvalue of 2.46 with a variance of 16.44%. The fourth factor has an Eigenvalue of 1.95 with a variance of 12.99%.
The fifth factor has an Eigenvalue of 1.10 with a variance of 7.37%. The overall factor gives a cumulative diversity proportion of 86.64%.
Keywords: Construct validity, Assessment, Eigenvalue, and Principal Component Analysis.
1. Introduction
In today's world, education is no longer a luxury but a necessity, and along with this importance comes a number of issues that call for a shift in both the quality and the organization of educational institutions. In order to do this, education must become more integrated with the other supporting sectors, which necessitates the enhancement of educational content. However, this is not enough since instructors or teachers are the ones who really give education in classrooms and schools. Teachers play an essential part in bridging the gap between what is now accessible in the form of curriculum and the requirements of the business sector and what is wanted to be available in the future.
Therefore, it is imperative that educators respond to the shifting issues and prepare themselves to meet the demands of the current moment.
Students are also other pivotal agent in educational setting. Students are living biological organisms that are always growing and maturing. Students, each of them, have their own unique potential, including their abilities, interests, and accomplishments, among other things. The value that is placed on a person's accomplishments continues to rise along with the level of sophistication of society. Educational institutions place a strong emphasis on high learning performance, healthy competition, and successful achievement on a variety of standardized examinations, including assessments of knowledge and aptitude. The school is one of the official educational institutions that are available. One of
the means that are utilized in the process of putting into action and developing education is the school itself. Hence, tests and numerous instruments are designed to measure the achievements and performances.
Thus, another viewpoint should be involved—which is the validity of the test/measurements. Cyril (2005) underlined that test validation is an attempt to guarantee that the data discovered properly or the data from the test analysis genuinely reflects the character of the student, not any other unrelated difficulties. This generates evidence to support the notion that the exam genuinely measures what it claims to assess in its most fundamental form. The creators of tests are obligated to present convincing justifications for the correctness of their instruments in measuring particular characteristics, as well as trustworthy data to back up the plausibility of these interpretative arguments (Haider, et.al, 2013). In a further illustration, this may be compared to the salute given by a defense attorney while they are defending their client in a courtroom. If, for example, a test claims that a student is capable of expressing thoughts in the form of a descriptive text, then the test truly does have the credibility to establish that claim, and other people should examine the student's abilities.
In order to validate certain aspects of certain assessments for teachers, actual literature and intricate methods are frequently relied upon. In point of fact, Hughes (2003) frames the majority of his propositions in terms of 'categories of validity,' as well as the dependability of test scores. In his most recent debate, Hughes (2003) offers his interpretation of a phrase that is known as consistency in scoring. As part of this effort, some of the fundamental approaches to analyzing consistency in rating and criterion- related validity are being presented to the instructors/teachers to acknowledge. The goal of this endeavor is to ensure that the instructor/teacher is able to comprehend the process of validating tests without experiencing any confusion. For example, in attempt of explaining the concept of the correlation coefficient to the instructor/teacher, the aim is to look at classroom assessment in terms of traditional categories such as linking the tests in order to establish the requirements for validating them. Despite the fact that the tests have a significantly complicated pedagogical purpose, this is the objective component. As part of the evaluation of the teaching and researching program, it is also necessary to take into consideration the results of tests of performance and progression designed to guide the creation of new lessons (McCowan & McCowan, 1999). This brings us back around to the beginning of the presentation of the scoring system that is supposed to be norm-referenced in nature.
Subsequently, despite the fact that it makes good use of a significant portion of the practical guidance offered in texts for educators, the distinction that exists between the classroom and large-scale testing is not taken into account because it has the potential to result in a great deal of confusion. When everyday usages of common testing are mixed together, this misunderstanding can have an effect on the daily assessment of tests as a large-scale definition. For instance, the term "reliability" may also signify
"trustworthiness": if an employee shows up to work every day, then the employee's attendance is dependable because the employer can trust that the worker will do so. If an employee misses work, then the employee's attendance is not reliable (Fransozi, 2007).
When it is narrowed down into Principal Component Analysis (PCA), it is anticipated that the Principal Component Analysis (PCA) method will be capable of reducing and eliminate factors that are less dominant or relevant in influencing a certain performance but have a sizable sufficient correlation to the formation of influencing factors with a total estimated ratio of variance covariance of 60%. This will make it simpler for educational institutions to further enhance the students' learning achievement, or research validity and reliability to be used as one measure of academic success (Jolliffe, 2002). In this study, principal component analysis (PCA) was performed to conduct an analysis of
research instruments that were used in an earlier investigation concerning the development of a learning environment as a result of the utilization of a specific instructional resource.
This method of analysis was employed because it provides an effective framework for doing further concession on data analysis. Both the raw data and the average of the raw data are taken into consideration for this selection. The covariance matrix is calculated, and then the results are used in the calculation of the eigenvectors and eigenvalues (Jhonson &
Wichern, 2007). The eigenvector with the highest eigenvalue was selected as the main component of the item validity as it demonstrated the most significant relationship between the characteristics of the data set. The data that is most significant are chosen by sorting the eigenvalues in ascending order, while the data that is least significant are thrown out. The data that normally takes up more span were reduced to take up less span to the result.
In brief, Principal Component Analysis is a method for selecting multiple variables (multivariables) from a data set. This method converts the original data matrix into a set of homogeneous combinations that are smaller but can accommodate a greater number of variations than the original data matrix (Hussain et al., 2015). The primary objective is to identify the most amount of the original data's variability with the fewest number of principal components as is humanly practicable (Dang, et al., 2016).
According to Jolliffe (2002), the procedure of Principal Component Analysis seeks to simplify and eliminate screening factors or indicators that are less dominant and less relevant without diminishing the intent and purpose of the original data from the random variable x (a matrix of size n x n, where the rows containing observations as many as n of random variables x are as follows). In order to achieve the following goal, the following steps were carried out: (1). constructing the covariance and variance matrices based on the observational data; (2). Finding the covariance matrix's eigenvalues and eigenvectors for the covariance matrix; (3). Determining the proportional value of the Principal Component expressed in percent; and (4). Calculating the factor weight, also known as the factor loading, based on the eigenvector using the equation, which states that Ax = λx—where λ stands for eigenvalue and x stands for eigenvector (Jhonson & Wichern, 2007).
This study is further considered significant as it gives deeper insights into the research instrument validity analysis. Hence, the research question generated is as follows:
“What are factors involved to obtain dominant elements influencing the centeredness in a learning environment by using Principal Component Analysis (PCA)?”
2. Method
This study was basically a qualitative study if examined on its data trend. A self- administered structured survey questionnaire—consisting of 10 questions—was used to collect primary data from ten lecturers who are teaching at higher education institutions.
They were chosen by employing purposive sampling with the following criteria: they are English lecturers, they have been teaching for at least 2 years, and they must also be involved in language and other educational research. Concerning the questionnaire instrument, Likert Scale was used in the questionnaire. The rating involved the respondents to decide whether the variable is “Not Suitable (1)”, “Less Suitable (2)”, “Fairly Suitable (3)”, “Very Suitable (4)” and “Extremely Suitable (5)”
The data were later analyzed using SPSS 22 involving PCA analysis. To generate meaningful interpretation, three-step analysis (data reduction, data display, and data verification) by Miles, Huberman, and Saldana (2014) were used.
3. Results and Discussion
The results are as elaborated in the following. First, it is presented the result of constructing the covariance and variance matrices based on the observational data as seen in Table 1.
Factor analysis, according to Field (2005), is effective for identifying clusters of related variables and hence suitable for reducing a huge number of variables into a more readily understood framework. The initial attempt at using factor analysis was to address some significant questions concerning the optimum sample size for performing and establishing the reliability of factor analysis (Field, 2005). As seen in the table above, the extraction value must be above 0.5 in order to be a factor for all variables. Since al potential variables are above 0.5, then all variables can be progressed into further factor analysis—total variance analysis which can be seen in Table 2.
From table 2 (as also supported by the Scree Plot), there are five factors in the initial
solution have eigenvalues greater than 1. Together, they account for almost 86.64% of the variability in the original variables. The factors extracted for further study are shown in table 2.
There are 10 factors dropped from the analysis as they have eigenvalue below 1. The rest 5 factors are later included into varimax rotattion to precisely see which instrument item each of the represents (Table 3).
In interpreting Table 3, the premise needs to be understood is that the variable with the highest value (0.5 or above) then enters the correlation into that factor and become the element of that particular factor. Subsequently, there are five factors and their elements as: Factor 1 has three components which are LC3, KC2, and KC 4; Factor 2 has five components which are LC4, KC3, KC5, AC2, and AC4; Factor 3: has two components which are KC1 and AC1; Factor 4 has three components which are LC5, AC3, and AC5; and Factor 5 has three components which are LC1 and LC5 (LC stands for Learner centered item, KC stands for knowledge-centered item)
4. Conclusion
The literature that supports investigating skills has been splintered and tenuously connected up until this point, and there has been a lack of a solid knowledge of the complicated linkages between the numerous instruments that have been made accessible.
In this specific investigation, a research instrument concerning three distinct types of centeredness in the learning process was measured. The use of principal component analysis, which has rigorously provided understanding into the complex structure and the relationship between the various competency areas, is the key contribution that this paper makes to the existing body of knowledge. This is manifested in the use of the principal component analysis. This puts together a basic framework that should guide the planning of teaching and researching in higher education, and it may also form the foundation for the design of tests and research instrumentation in pedagogical context.
References
Cyril, W. J. 2005. Language Testings and Validations. Hampshire: McMillan Publishers.
Dang, T., Pham, T., Tran, H. & Van, Q. 2016. Using Dimension Reduction with Feature Selection to Enhance Accuracy of Tumor Classification. International Conference on Biomedical Engineering, pp.14-17.
Fransozi, R. 2007. Content Analysis: Objective, Systematic, and Quantitative Description of Content. CA: Sage Publication.
Haider, Z., Latif, F., Akhtar, S., & Mushtaq, M. 2013.Evaluation of English achievement test: A comparisonbetween high and low achievers amongst selectedelementary school students of Pakistan. African Journal of Adult Education, Vol. 1, No.3,40- 47.
Hughes, A. 2013. Testing for Language Teachers. Cambridge:Cambridge University Press.
Hussain, H., Quazilbash. N.Z., Bai. S. & Khoja, S. 2015. Reduction of Variables for Predicting Breast Cancer Survivability Using Principal Component Analysis.
International Conference on Computer-Based Medical Systems, pp. 131-134.
Johnson, W.A. & Wichern, D.W. 2007. Applied Multivariate Statistical Analysis. 6th Edition. Pearson Prentice Hall: New Jersey.
Jolliffe, I.T. 2002. Principal Component Analysis. 2nd Edition. Springer-Verlag: New York.
McCowan, R., and S.McCowan. 1999. Item Analysis for Criterion-Referenced Tests. NY:
Research Foundation of SUNY press.
Miles, M.B., Huberman, A.M. and Saldana, J. (2014). Qualitative Data Analysis A Methods Sourcebook. SAGE.
