There are many types of performance analysis investigation and indeed investigations that involve performance analysis in conjunction with other disciplines. This means that there are no standard experimental or observa- tional designs that can be used, and some investigations may be the fi rst to
use a particular research design. Therefore, an ability to understand when different statistical tests should be applied, how they should be used, how tests can be tailored for particular research designs and how statistical results can be presented are essential for the successful performance analysis researcher. The analysis depends on the number of independent factors involved in the study, the types of samples of data involved, the scale of measurement and distributions of the dependent variables and above all, the purpose of the study.
The purpose of the study should be understood before collecting data in quantitative research. Specifying formal hypotheses to be tested allows the gathering and analysis of data to be driven by the goal of the study which is to answer the research question. As explained in Chapter 4, hypotheses are specifi ed in terms of the variables that are used in the study. It is, therefore, necessary to have an understanding of the variables involved in the study as well as their scales of measurement. Where categorical variables are used to form samples, the types of sample also need to be understood when forming hypotheses and when conducting the eventual analysis. This chapter will now briefl y recap on the different types of sample that can be compared.
Samples can be independent samples or related samples. Independent samples are samples of data drawn from different participants or matches that are being compared; examples are data from male or female partici- pants or from players of different positional roles. Because the samples contain independent data, it is not necessary for the size of the samples to be the same. Related samples are samples of data that are related to the same participants or matches; examples are player performances when playing at home and when playing away from home, or player performances in the four different quarters of a netball match. It is necessary for related samples to contain the same number of values because they are matched samples that represent different conditions for the same subjects.
The purpose of the study could be to test the relationship between vari- ables or it could be to test for differences between samples. Figure 8.1 shows the different types of test that can be used for these different pur- poses when different types of samples are used. The decision tree shown in Figure 8.1 is not exhaustive and only represents those tests of associations that are between two variables and tests of differences involving single independent factors. The tests of association may involve variables with different scales of measurement. For example, we may wish to test the rela- tionship between a nominal variable and an ordinal variable. In such a situ- ation we should use the chi-square test of independence because an ordinal variable is also a nominal variable (with ordered values) but a nominal variable is not always an ordinal variable. Similarly, if we wished to test the association between an ordinal variable and a numerical variable (interval or ratio scale measure), we would use either Kendall’s τ or Spearman’s ρ because numerical variables are also ordinal variables but not all ordinal variables are numerical variables.
Figure 8.1 presents two different tests of difference between samples in each of the four situations shown. The decision as to whether to use the parametric test or the alternative nonparametric test depends on the scale of measurement of the dependent variable and its distribution. These tests of differences can only be used with ordinal and numerical scale variables to test differences between the medians and the means respectively. Where the dependent variable is an ordinal scale measure, a nonparametric test should be used. Nonparametric tests use the rankings of values rather than the values themselves when calculating the test result. Ordinal values can be transformed into ranks in the same way as numerical values can be trans- formed into ranks.
Where the dependent variable is measured on a numerical scale, the deci- sion to use a parametric test or a nonparametric test depends on whether the data satisfy the assumptions of the test. Parametric tests are called paramet- ric tests because the results are calculated using the sample parameters (means, standard deviations and sample sizes). They are more powerful
Nominal variables Ordinal variables Testing
relationships between variables
Testing differences
between samples
Numerical variables
Comparing 2 samples
Comparing 3+ samples
Comparing 2 samples Independent
samples
Related samples
Comparing 3+ samples
Independent samples test
One-way ANOVA
Paired samples t test
Repeated measures ANOVA
Mann Whitney
U test Parametric
tests
Nonparametric tests
Kruskal Wallis H
test Wilcoxon
signed ranks test Friedman
test Chi-square test
of independence Kendall’s or Spearman’s
Pearson’s r
Figure 8.1 Decision tree of statistical tests
than nonparametric alternatives that lose information when transforming values into ranks. Therefore, where researchers can validly use parametric or nonparametric tests, they should always choose to use the parametric test. There are three main assumptions of parametric tests, although some tests have additional assumptions:
1. the dependent variable is measured on an interval or ratio scale;
2. the dependent variable is normally distributed; and
3. the variance of the dependent variable is similar between the different samples being compared.
The fi rst of these assumptions has already been addressed as we are only considering the use of parametric tests for numerical scale dependent vari- ables. Simple inspection of the defi nition of the dependent variable will determine whether it is a numerical scale variable or not. There are tests for the normality of the dependent variable and tests for equality of vari- ances, which will be outlined later in this chapter. Where a numerical var- iable passes these tests, parametric tests can be validly applied to the data.
Where the dependent variable is not normally distributed or where the samples being compared have differing variances, it is necessary to use an alternative nonparametric test. The nonparametric tests are ‘distribution free’, meaning they can be used with any data that can be ranked. There are some assumptions that apply to nonparametric tests; for example the Kruskal Wallis H test requires that there are at least fi ve values in each sample being compared.
The same research question can be examined using different research designs. This often depends on the unit of analysis being used in the study.
For example, we may wish to compare home and away performances (using some dependent performance indicator) in a sport by using any of the fol- lowing designs:
Comparing home and away performances within a given set of matches.
•
Here, the unit of analysis is the match and we would be comparing home and away team performances as related conditions within matches. A paired samples t-test could be used if the data satisfi ed the necessary assumptions. The performance of a team is infl uenced by the performance of the opponent, so in this research design it would not be appropriate to treat home and away performances as independent samples.
Comparing teams’ mean performances in matches where they play at
•
home and where they play away from home. Here, the unit of analysis is the team and the home and away conditions are used to form related samples with each team’s mean home match performance and mean away match performance being compared. Again a paired samples t-test could be used if the data satisfi ed the necessary assumptions.
Comparing a set of home performances with a set of away perform-
•
ances where different sets of teams are involved in the home perform- ances and the away performances. Here the unit of analysis is the match performance, as the same team may be involved in multiple matches within the sample of home performances as well as multiple matches within the sample of away performances. The two samples would be considered as two independent sets of performances being compared.
Therefore, an independent samples t-test could be used if the data satis- fi ed the assumptions of the test.
Comparing the proportion of events that are performed positively at
•
home with the proportion that is performed positively away from home.
In this example, there may not be enough matches to use the perform- ance indicator of interest. For example, our performance indicator might be the percentage of passes that are successful and we may only have data from four matches involving a total of eight teams. A way of analysing the data is to make the event the unit of analysis rather than whole performances or matches. The events are classifi ed using a nominal variable of two values (positive and negative) and the propor- tion of positively performed events could be compared between home and away performances using a chi-square test of independence.
These different ways of investigating the same research question mean that the researcher should consider the volume of data to be gathered in each case, the feasibility of being able to access this data, and how much time and effort would be required. Once a particular research design is chosen, the researcher develops the system to gather the data required and may have to process the data in order to produce the specifi c values to be entered into a statistical analysis package.