CHAPTER 5: CONCLUSIONS, LIMITATIONS AND
5.2 ACHIEVEMENT OF THE RESEARCH OBJECTIVES
5.2.1 GENERAL OBJECTIVE
The general objective of this study was to determine whether block- or random-item format delivers the best factorial replication within the framework of the SAPI. This was done by comparing the results obtained by administering block- and random-versions of the SAPI.
This objective was designed to assist with the overall objective of the SAPI project, which is to guarantee that the SAPI accommodates all South African cultures on a proven scientific platform. Four aspects were investigated in order to meet this general objective, namely qualitative interpretability, item similarity and logical interpretability; replication of the conceptual qualitative SAPI model; reliability of the factors and clusters; and existing literature on the relevant subject.
Based on the findings relating to these four aspects it was determined that the block response format was marginally more suitable for the SAPI. The block response set had fewer problematic items, better factor loadings that resulted in better item interpretability and similarity, and slightly higher internal reliability than the random response set. In addition, it was determined that the random response set best replicated the conceptual qualitative personality structure across all nine clusters with one additional cluster when compared to the block response set. However, when analysed on an overall factorial level,
the block response set replicated two factors more than the random response set. There were thus no major replication differences between the random-and block-response sets.
Moreover, all the other differences were insignificant relating to the four aspects listed above as comparability points, when considering the sample size of the study and the number of items within the SAPI. Therefore the findings indicated that the random response set can also be used for the SAPI as the differences between the results of the two response sets were minor.
In addition, the findings in the present study are confirmed by previous research regarding personality. Ortner (2004) identified a German article by Rost and Hoberg (1997) that stated that no differences were identified in the construct validity of the factor structure when using either block- or random-response forms. However, other researchers have reported higher average scores when using the block item format. Baehr (1953), Schriesheim and DeNisi (1980), Schriesheim (1981), Schriesheim et al. (1989) also investigated the effects of item grouping and could not find significant differences between item blocking and randomization.
5.2.2 SPECIFIC OBJECTIVES
This section discusses the three specific research objectives presented in Chapter 1 (see section 1.5). This discussion is integrated with a discussion of the analysis of factor structure, including qualitative interpretability, item similarity and logical interpretability.
This factor analysis strategy is included in the discussion as it contributed to the final recommendations of this study.
The first specific research objective was to conduct a thorough literature study on block- and random-item format. Although limited research was available on the subject of item sequencing in relation to the difference between block- and random-item formatting, a comprehensive literature review was conducted using available sources (Chapter 2, section 2.7). In short, the advantages and disadvantages of each item format were discussed and presented in table format. The immediate impression when analysing these tables was that the block item format presents with more advantages and less disadvantages than its counterpart. However, it should be kept in mind that the literature
on block advantages is strongly based on autistic (Stewart et al, 2009) and dyslexic (Kelly et al., 2002) samples and is therefore not related to a normal population.
The second specific research objective was to determine whether the block- or random- item format best replicated, or was best representative of, the preliminary qualitative personality factor structure of the SAPI. The replication of the conceptual qualitative SAPI model within the block- and random-response sets showed that four of the clusters replicated the same number of factors in both block- and random-response sets. These clusters were Facilitation with two replicated factors; Soft-Heartedness with three replicated factors; Extraversion with three replicated factors; and Openness with one replicated factor. Two clusters replicated more factors in the block response set, namely Intellect with three replicated factors and Emotional stability with five replicated factors.
Three clusters replicated more factors in the random response set, namely Conscientiousness with four replicated factors, Integrity with two replicated factors and Relationship Harmony with four replicated factors. Based on the analysis in Chapter 4 (see section 4.3), it was determined that the random response set best replicated the conceptual qualitative personality structure across all nine clusters. However, when analysed on an overall factorial level, the block response set replicated two factors more than the random response set. Therefore no major replication differences are applicable between the random– and block-response sets.
The third specific research objective involved identifying the difference between the block- and random-item formats in order to determine which format is more reliable for measuring personality specific to the SAPI. In evaluating the reliability of the two response sets, specific cut-offs were used (see Chapter 3) to ensure clearer interpretation and classification of scores. In the final step of the analysis, the Cronbach alpha coefficients of all nine clusters for both the random- and block-response sets were identified (see Chapter 4)
For the purpose of the study, reliability scores were classified as high if they were equal to or above 0.80 and as acceptable if they were equal to or above 0.60. These cut offs are deemed suitable for exploratory studies such as this one (Cortina, 1993; Field, 2009;
Robins et al., 2007). Reliability scores lower than 0.60 were not tolerated and were
deemed unacceptable (Maree, 2010). The Cronbach alpha coefficients for the block response set ranged from 0.54 to 0.92, whereas the Cronbach alpha coefficients for the random response set ranged from 0.47 to 0.90. Eight of the nine clusters for both the random- and block-response sets had reasonable internal consistency. For both the random- and block-response sets the Emotional Stability cluster had a Cronbach value below 0.60 and was therefore deemed unreliable. The analysis showed that the block response set had higher overall Cronbach alpha coefficients for all except two of the clusters (Relationship Harmony, which showed the same Cronbach value for both response sets and Soft-Heartedness).
An additional strategy for analysis (which was not one of the specific objectives noted in Chapter 1), namely the analysis of factor structure, was also undertaken. This was done by looking at and comparing the qualitative interpretability (themes), item similarity and logical interpretability of the factor loadings. Based on the analysis in Chapter 4 (see section 4.3), it was determined that the block response set portrayed clearer themes and logical interpretability per factor than the random response set.