CHAPTER 3 RESEARCH METHODOLOGY
3.2 Quantitative Research
3.2.5 Data Analysis
Prior to the analyses, data were screened for entry errors, missing values, and outliers that may impair data analysis. Validity and reliability were also tested. Three categories of missing values are discussed in the literature: 1) values missing completely at random, 2) values missing at random, and 3) values systematically missing. Five to ten percent missing data on a variable may be judged small (Cohen et al., 2003), while data with 40% missing values on a variable is considered high (Raymond & Roberts, 1987).
The online survey method adopted in the pilot, and main survey, resulted in few missing values overall. Tabachnick & Fidell (2019) suggested that cases with missing values should be deleted to prevent overestimation. In dealing with missing values, cases with more than 10% missing values should be deleted from the dataset (Jin, 2010).
Therefore, five cases in the pilot and thirty from the main survey were discarded.
3.2.5.1 Exploratory Factor Analysis
Exploratory factor analysis (EFA) is a statistical procedure used to reduce a large number of observed variables to a small number of
"factors/components", reflecting that the clusters of variables are in common (Ul Hadia
et al., 2016). Recently, EFA was applied for a wide range of applications (Taherdoost et al., 2014), e.g., assessing the motivation (Morris, 2001). It is a useful tool for investigating the relations among observed variables and a small number of underlying factors.
EFA was first performed to determine the underlying dimensions that explain common variance in the sample. The EFA explores the data and provides information about how many factors are needed to best represent the data (Hair et al., 2010). It was undertaken through a principal component factor analysis approach with the VARIMAX orthogonal rotation using the Statistical Package for Social Sciences (SPSS) 21.0.
A measure of sampling adequacy (MSA) test was performed using EFA to ensure that the variables are sufficiently intercorrelated to produce representative factors. In the SPSS software, the MSA is measured by the value of Kaiser-Meyer-Olkin (KMO), and the factorability of the correlation matrix is assumed if Bartlett’s test of sphericity is statistically significant (p < .05) and the MSA values is greater than 0.50 (Hair et al., 2010).
Furthermore, multicollinearity can be detected by examining the determinant of the R-matrix (R-matrix > 0.00001) (Field, 2018). In extracting factors, Kaiser’s criterion of eigenvalue greater than one and total variance explained above 60% was utilized (Hair Jr. et al., 2014). Items with communalities lower than 0.50 were removed for not having sufficient common correlations with other items (Hair et al., 2010).
Concerning a number of measured variables, Izquierdo, Olea, & Abad (2014) suggested that at least three indicators are needed for the statistical identification of a factor, although more indicators are preferable, while, Fabrigar, MacCallum, Wegener, & Strahan (1999) recommended four to six indicators per factor; this depends on the design of the study (Tabachnick & Fidell, 2019). Nevertheless, Hair Jr. et al.
(2014) recommended that factors with fewer than three indicators should be avoided for confirmatory factor analysis.
3.2.5.2 Confirmatory Factor Analysis
Confirmatory factor analysis (CFA) is frequently used in scale development and validity analysis or in verifying a predetermined structure (Sözbilir,
2021). CFA attempts to confirm hypotheses and uses path analysis diagrams to represent variables and factors (Child, 2006). The researcher must specify five elements: the latent constructs, the measured variables, the item loadings on specific constructs, the relationships among constructs, and the error terms for each indicator (Hair Jr. et al., 2014).
CFA was conducted to evaluate the measurement model of internal consistency reliability and indicator reliability (composite reliability), convergent validity, and discriminant validity (Sözbilir, 2021). It enables the researcher to test how well the measured variables represent the construct (Hair et al., 2010). The main objective of using CFA is to test a measurement model and assess the proposed internal relationship based on the hypotheses incorporated into the conceptual research framework ( Brown, 2015). When CFA results are combined with construct validity tests, the researchers can better understand the measured quality (Hair Jr. et al., 2014).
Concerning the construct validity, Hair et al. (2010) suggested that
a) Standard loading estimates should be .5 or higher, ideally .7 or higher.
b) Average variance extracted (AVE) should be .5 or greater to suggest adequate convergent validity.
c) AVE estimates for two factors also should be greater than the square of the correlation between the two factors to provide evidence of discriminality validity.
d) Construct reliability should be .7 or higher to indicate adequate convergence or internal consistency.
3.2.5.3 Structural Model
After adequate measurement and construct validity were established using CFA, SEM was conducted to test the structural model. In this study, CFA and SEM were performed using Mplus 6. The structural model represented the hypotheses of interests.
Assessment of overall model fit
Using three to four model fit indices provides adequate evidence of model fit (Hair et al., 2010). Table 3.6 provides guidelines for using fit indices in different situations when the number of respondents is larger than 250.
Table 3.6 Guidelines for Using Fit Indices
Fit indicies M ≤ 12 12 < M < 30 M ≥ 30
Χ2 Insignificant p-value even
with good fit
Signifincant p-value expected Signifincant p-value expected
RMSEA Values <.07 with CFI of .97
or higher
Values <.07 with CFI of .92 or higher
Values <.07 with CFI of .90 or higher
SRMR Biased upward; use other
indices
.08 or less (with CFI above .92) .08 or less (with CFI above .92)
CFI or TLI .95 or better Above .92 Above .90
Note: N>250, M = number of observed variables, RMSEA = Root mean square error approximation, SRMR = Standardized root mean squared, CFI = Comparative fit indices, TLI = Tucker Lewis index.
Source: Hair et al., 2010
In this study, there are 501 respondents with more than 30 observed variables, the appropriate criterion of the model fit indices suggested by Hair et al. (2010) is summarized in Table 3.7.
Table 3.7 Criterion of the Model Fit Indices
χ2 /df RMSEA SRMR CFI TLI
The target of criterion (Hair et al., 2010) 3 < 0.07 < 0.08 > 0.90 > 0.90
Note: RMSEA = Root mean square error approximation, SRMR = Standardized root mean squared, CFI = Comparative fit indices, TLI = Tucker Lewis index.