Chapter 3: Methodology
3.6. Validity and Reliability
3.6.1. The Questionnaire
Furthermore, Helmreich et al. (1981) showed that the PAQ had good criterion and predictive validity in a discriminant analysis as gender classification was 80% correct for high school students, 77% for university students and 81% for parents, furthermore the chi-squared values were all significant at the level of α=0.001. The PAQ is comparable to another measure of gender conformity, Bem’s sex role inventory, as it was similarly able to correctly classify gender, therefore indicating the concurrent validity of the PAQ. According to Loewenthal (2001) concurrent validity refers to how different scales claiming to measure the same thing compare.
Although the psychometric analyses conducted in 1981 were “satisfactory and consistent”, these analyses were conducted more than twenty years ago. Therefore, it is reasonable to assume that gender roles and norms may have changed somewhat. For this reason, estimates of internal consistency were calculated for both sexes for the M and F subscales for this study. The Cronbach’s alpha for each sex on each subscale is reported in table 5 below.
Table 5: Cronbach’s alpha for males and females for both the M and F PAQ subscales
PAQ subscale Males Females
M .840 .262
F .880 .644
From the internal consistencies reported above it would appear that the sample of women in this study were less consistent in their responses to the items regarding the M subscale.
However, men were more consistent in their responses to both M and F subscales in comparison to the psychometrics reported by Helmreich et al. (1981). The internal consistency of female responses regarding the F subscale was moderately satisfactory however not as impressive as male consistency. This may suggest that since the development
Table 4: Cronbach’s alpha for F and M PAQ subscales from the psychometric analysis conducted by Helmreich et al. (1981)
PAQ subscales
High school respondents University Students Parents
Males Females Males Females Males Females
M .67 .71 .76 .73 .78 .77
F .72 .73 .76 .73 .80 .79
and testing of the psychometric properties of the PAQ in 1981, women, more so than men have changed their perception of gender roles in particular to the conformity of masculine gender roles. However, the values of internal consistency for the PAQ for this study were calculated from small sample sizes and therefore may be inaccurate in their estimation of criterion and predictive validity. Larger samples would need to be obtained for greater accuracy.
A chi-square analysis was conducted comparing sex, SO and groups of sex by SO to further assess the association of the PAQ with sexuality and gender role conformity, the results of this analysis are reported in chapter 4.
Ranking scale
In this study as compared to previous t-shirt studies (Singh & Bronstad, 2001; Thornhill &
Gangestad, 1999; Thornhill, et al., 2003; Trouton, et al., 2012; Wedekind, et al., 1995), participants were asked to rank stimuli items in order of attractivness. Previous studies used Likert scales or rating scales as a measure for participants to indicate the attractiveness of the scent and or visual stimuli. However, the researcher expected that response bias may lead the judges in this study to consistently rate the scent of the t-shirts negatively because of the unfavourable connotations associated with BO. Therefore, by asking participants to rank the stimuli, it forced respondents to indicate their preference, and minimises the influence of response bias or consistent “nay saying”. To assess the inter-rater-reliability and consistency of the rankings a Kendall’s coefficient of concordance 𝑊̃ was used. The statistic 𝑊̃ was also used in the main analysis as a measure of effect size for the Friedman’s test as well as to assess pairwise comparisons, therefore the reliability statistics of this scale will not be presented here, but will be presented and discussed in chapter 4.
The criterion validity of this the ranking scale was influenced by previous studies although not entirely dictated by them. In this study the criteria or measures for ranking the visual and scent stimuli were attractiveness and masculinity. In previous t-shirt studies, the criteria for rating the stimuli were often threefold and included ratings of pleasantness, sexiness and or intensity (Trouton, et al., 2012; Wedekind, et al., 1995; Singh & Bronstad, 2001; Thornhill &
Gangestad, 1999). However, Wedekind et al. (1995) found that using both pleasantness and sexiness was redundant as they correlated very highly, therefore only the measure of pleasantness was reported on. It was deemed by the researcher of this current study that
“attractiveness” was a cumulative term for both pleasantness and sexiness. In addition, the term “attractiveness” is defined as how attractive one individual or object seems to another, whereas, the term pleasantness and sexiness both may not necessarily imply attraction specifically.
It was also necessary to establish whether masculinity as a construct was judged similarly to what previous research suggests. According to Thornhill and Gangestad (1999) masculinity is precipitated by a pronounced jaw, facial hair and a heavier set brow, which would be evident in the photographs. Measures of reliability such as 𝑊̃ would indicate whether the features that are deemed masculine in the literature are validated by the consensus of the sample.
Furthermore, whether individuals consider visual masculinity (VM) similarly to scent masculinity (SM) and how these rankings of masculinity fared against rankings of attractiveness, especially for women. Therefore, it was necessary to include both measures of attractiveness and masculinity to provide criterion validity to the study.