R Output
mean sd IQR 0% 25% 50% 75% 100% data:n data:NA Female 123.971 8.27997 9 107 120 124 129 144 35 0 Male 142.600 21.27401 32 99 125 142 157 192 25 1
2.6 Quality Assurance, Data Distribution, and Tests for
Test for Normality is worded when interpreting the p-values, significance levels, and outcomes from a Shapiro Test for Normality.
It is common to focus on a p-value of either p <= 0.05 or p <= 0.01 when considering normality tests, such as the Shapiro Test for Normality. Remember that due to the way the Shapiro Test for Normality is worded in the affirmative:
• Lower calculated p-values provide evidence against the null hypothesis. If the calculated p-value is less than or equal to the criterion p-value (e.g., significance level), the rules-based decision is to reject the null hypothesis and to declare that the data do not follow a pattern of normal distribution.
• Higher calculated p-values fail to provide evidence against the null hy- pothesis. If the calculated p-value is greater than the criterion p-value (e.g., significance level), the rules-based decision is to accept (e.g., fail to reject) the null hypothesis and to declare that the data seem to follow a pattern of normal distribution.
As demonstrated throughout this text, many different types of figures and text- based output are used to offer a sense of the data, overall. A key interest when viewing the many figures and statistics on central tendency is the con- cern over normal distribution for numeric object variables or Lbs in this lesson.
Normal distribution is important in that many inferential tests are only used, appropriately, if the underlying data exhibit normal distribution. Of course, compromised decisions are often made and it not uncommon to see the appli- cation of statistical tests that call for the use of normally-distributed data with data that do not follow a normal distribution pattern—certainly not a perfect normal distribution pattern. The question, then, is how much deviation away from normal distribution is acceptable before a nonparametric approach to sta- tistical analysis is necessary. This is always a judgment call, but the Shapiro Test for Normality will at least provide a sense of data distribution. From that outcome, it can be determined whether it is accepted that the data exhibit a reasonable pattern of normal distribution or if the normality of the data should be questioned.
Apply the shapiro.test() function against CPIIISecLbsGen.df$Lbs, overall, to address these first-level concerns. For now, there is no focus on factor-type object variable breakouts: CPIIISecLbsGen.df$Section (e.g., AM or PM) or CPIIISecLbsGen.df$Gender (e.g., Female and Male).
R Input
shapiro.test(CPIIISecLbsGen.df$Lbs)
R Output
p-value = 0.000578
The calculated p-value (0.000578) for CPIIISecLbsGen.df$Lbs is less than 0.05, providing evidence that normal distribution is not evident for the ob- ject variable CPIIISecLbsGen.df$Lbs, overall. This finding needs to be con- sidered in the future when deciding on whether to use a parametric test or a test when using CPIIISecLbsGen.df$Lbs. However, going beyond the use of CPIIISecLbsGen.df$Lbs at the overall level of comparison is deviation away from deviation away from normal distribution for CPIIISecLbsGen.df$Lbs a concern for the two levels of Section (e.g., AM and PM) and for the two levels of Gender Male)? It is now known thatCPIIISecLbsGen.df$Lbs (e.g., weight) does not follow a normal distribution pattern, but:
• Is it possible that the weight of AM students follows a normal distribution pattern, regardless of the distribution pattern of weight for PM students?
• Is it possible that the weight of PM students follows a normal distribution pattern, regardless of the distribution pattern of weight for AM students?
• Is it possible that the weight of Female students follows a normal distri- bution pattern, regardless of the distribution pattern of weight for Male students?
• Is it possible that the weight of Male students follows a normal distribu- tion pattern, regardless of the distribution pattern of weight for Female students?
The RVAideMemoire::byf.shapiro() function is an excellent choice to examine these questions on normality by breakouts of factor-type object variables. Ob- serve below how the RVAideMemoire::byf.shapiro() function provides a simple and convenient summary of normality p-values by individual breakouts of a factor-type object variable.8
8Normality tests examine distribution patterns, not difference and not association. The Shapiro Test for Normality makes no attempt to determine if there is a statistically significant difference (e.g., p <= 0.05 or p <= 0.01) between AM and PM subjects in terms of weight.
The Shapiro Test for Normality makes no attempt to determine if there is a statistically significant difference (e.g., p <= 0.05 or p <= 0.01) between Female and Male subjects in terms of weight. The Shapiro Test for Normality and more specifically for this lesson, the RVAideMemoire::byf.shapiro() function, is only used to examine normality by declared breakout groups.
R Input
install.packages("RVAideMemoire", dependencies=TRUE)
library(RVAideMemoire) # Load the RVAideMemoire package.
help(package=RVAideMemoire) # Show the information page.
sessionInfo() # Confirm all attached packages.
R Input
# Normality test of Lbs overall
RVAideMemoire::mshapiro.test(CPIIISecLbsGen.df$Lbs)
R Output
p-value = 0.000578
# Compare to use of the shapiro.test() function.
R Input
# Normality test of Lbs by Section
RVAideMemoire::byf.shapiro(Lbs ~ Section, data=CPIIISecLbsGen.df)
R Output
W p-value AM 0.9560 0.244497 PM 0.9024 0.009616 **
R Input
# Normality test of Lbs by Gender
RVAideMemoire::byf.shapiro(Lbs ~ Gender, data=CPIIISecLbsGen.df)
R Output
W p-value Female 0.9865 0.9367 Male 0.9823 0.9261
Use the ggplot2::ggplot() function to produce Q-Q plots of the numeric object variable Lbs (e.g., weight), overall, and then by breakouts of each of the two factor-type object variables (e.g., Section and Gender).9 Use these figures as a visual reinforcement of outcomes from application of the Shapiro tests (Fig.2.4).
R Input
# Weight (Lbs), Overall QQLbs <-
ggplot2::ggplot(CPIIISecLbsGen.df, aes(sample=Lbs)) +
stat_qq(color="red") +
stat_qq_line(color="blue", size=1.75) + ggtitle("
Weight (Lbs) QQ-Plot and QQ-Line,\nOverall\n") + labs(x = "\nTheoretical", y = "Weight (Lbs)\n") + theme_bw()
# Weight (Lbs) by Section (two breakouts) QQFacetLbsSection <-
ggplot2::ggplot(CPIIISecLbsGen.df, aes(sample=Lbs)) +
stat_qq(color="red") +
stat_qq_line(color="blue", size=1.75) + facet_grid(. ~ Section) +
ggtitle("
Weight (Lbs) QQ-Plot and QQ-Line,\nby Course Section\n") + labs(x = "\nTheoretical", y = "Weight (Lbs)\n") +
theme_bw()
# Weight (Lbs) by Gender (two breakouts) QQFacetLbsGender <-
ggplot2::ggplot(CPIIISecLbsGen.df, aes(sample=Lbs)) +
9Similar to other terms that have multiple means of expression, throughout this text expect to see the terms Quantile-Quantile, Q-Q, and QQ to describe this graphical image.
stat_qq(color="red") +
stat_qq_line(color="blue", size=1.75) + facet_grid(. ~ Gender) +
ggtitle("
Weight (Lbs) QQ-Plot and QQ-Line,\nby Gender\n") + labs(x = "\nTheoretical", y = "Weight (Lbs)\n") + theme_bw()
R Input
gridExtra::grid.arrange(
QQLbs,
QQFacetLbsSection,
QQFacetLbsGender, ncol=3)
Figure 2.4: Weight Q-Q plot breakouts by section and gender
Overall, the Q-Q plots seem to reinforce the statistical outcome that there is overall deviation away from normal distribution. At a more finite level of observation for the breakouts, notice how the greatest degree of deviation away from normal distribution seems to be the weights above 150 pounds in the afternoon (e.g., PM) course section. Here is where the red circles, indicating weights of specific subjects, go farthest away from the straight fit line.
Quality Assurance is attempted throughout this lesson, by generating graphical images of the relationship between and among data and also by providing de- scriptive statistics and measures of central tendency between and among data.
Further, normality has been addressed by examining distribution patterns of the measured object variable, overall (e.g., Lbs), and then by breakouts of the factor-type object variables (e.g., Section, AM and PM; Gender, Female and Male).
As a general statement, it was observed that normal distribution of the numeric object variable was not evident in the dataset associated with this lesson, at the overall level of comparison. Using p <= 0.05 as the criterion value of ac- ceptance or rejection of each normal distribution Null Hypothesis, overall, and by breakouts:
• Overall, the data for CPIIISecLbsGen.df$Lbs do not follow a pattern of normal distribution (overall calculated p-value = 0.000578).
• When looking at normality by Section, students in the morning (e.g., AM calculated p-value = 0.244497) section had weights that followed nor- mal distribution, whereas students in the afternoon (e.g., PM calculated p-value = 0.009616) section had weights that did not follow normal dis- tribution.
• When looking at normality by Gender, both Female students (calculated p-value = 0.9367) and Male students (calculated p-value = 0.9261) had weights that followed normal distribution.
These findings, overall and by breakouts, must be considered when deciding which test(s) to use for either tests of difference or tests of association. Forward looking, should a useful test of difference, such as Student’s t-Test for Indepen- dent Samples be avoided merely because a few students in the afternoon course section possibly impacted normality? There is no one and only one answer to this question. Careful thought and consultation with others (e.g., colleagues, editors, peers, etc.), as well as discovery from the published literature, will pro- vide insight into the best course of action regarding findings of normality and eventual use of these findings on later statistical testing.