Once the student has formulated a research question and they know ‘what’
the project is investigating, the next step is to think about ‘how’ the project can be done. This involves devising methods in suffi cient detail for an ethics committee to be able to make a decision on the project. The following steps are used in the design of a typical manual notation project where data col- lection is restricted to publicly broadcast performances:
Consider the purpose of the study, the hypotheses and sketch the even-
•
tual format of the results. This involves thinking forwards and back- wards at the same time: forwards to the eventual results and backwards to the purpose of the study.
Identify the information to be derived from the manual notation
•
system.
If the variables to be used have not already been defi ned during the
•
process of specifying the research question described in Chapter 4, then they should be defi ned when devising the methods.
The raw data about events occurring within performances that need to
•
be recorded in order to produce the information required should be identifi ed.
Once the student is aware of the data that need to be gathered, a system
•
can be developed to allow these data to be recorded. The system will contain specifi cally designed forms to allow the necessary data to be recorded as simply and effi ciently as possible, in a manner that also eases the process of determining system output information from the raw data.
The system must then be pilot tested using a suffi cient period of a match
•
to ensure that it is usable and effi cient and that the data required are able to be observed from the broadcast coverage of the performances.
The system may have to be revised several times until the student and supervisor are happy with it.
It is necessary to demonstrate that the system produces results that are
•
independent of an individual user’s perception of behaviour during sports performance. This requires an inter-operator reliability study to be done. The student must train another operator to use the system and then the system must be used independently by the student and the other operator to analyse the same performance. The performance
analysed may be a whole match, a part of a match or indeed it may be necessary to include more than one performance in the data for the reli- ability test. The two independent observations are compared to gauge the level of reliability of the system. Where the system is unreliable for some or all variables being assessed, it is necessary to either revise the system or provide additional training to the observer and redo the reli- ability study.
Once the system has been fi nalised, the student should consider the
•
main study of the research project. There is a balance between the number of performances that are needed to answer the research ques- tion and the number of performances that can be analysed within the time available for the research project. This can be discussed, because after piloting and reliability testing, the student will be aware of how long is required to analyse each performance. In extreme cases, it may be necessary to modify the system again to increase the number of per- formances that can be included in the study or to make the system cover more areas of performance as the project in its current form may not be considered ambitious enough for the student’s programme of study.
The data are then collected using the manual system, with individual
•
match data processing being done as soon as possible after each match and an electronic data sheet of resulting match information is popu- lated. This data sheet should be backed up periodically and the raw data collection forms should be kept safe for verifi cation purposes.
The data are then transferred to a statistical analysis software package
•
where the necessary statistical tests are performed. These systems usually generate a great deal of information, so the student must extract those descriptive and inferential statistics required to produce the results chapter. This is discussed in Chapter 8.
The methods chapter of the dissertation should be written up in the past
•
tense as the eventual report will be describing research that has been done. It is a good idea for the student to make notes during system development and pilot testing on any issues that can be described in the methods chapter as these might be forgotten if the methods chapter is being written up after the data have been collected and analysed.
Projects that combine the development and use of manual notational analy- sis with other techniques will still have to go through the above stages. The difference is that the proportion of dissertation time to be devoted to the notational analysis and other elements needs to be decided when designing the project. There are additional stages for the non-notational analysis activities that are part of the study. If qualitative techniques are being used, they could be applied before, after or in parallel with the notation analysis part of the study. This will have implications for the scheduling of activities within the project plan. The qualitative techniques themselves involve activ- ities that need to be scheduled within the overall research plan. For example,
an interview study would involve the preparation of an interview guide, pilot interviewing, establishing trustworthiness of data, recruiting partici- pants, conducting the interviews as well as transcribing and analysing the interviews. The complementary use of notational analysis and qualitative techniques will hopefully not be done in a disjointed fashion. Therefore, there may be an additional task of integrating the two sets of fi ndings, although this is typically embedded within the qualitative analysis that would be explaining the quantitative results. The integration of quantitative and qualitative aspects continues within the discussion.
If a computerised performance analysis system is being developed, whether using commercial sports analysis packages or not, the stages listed previously are still used except that code templates are developed instead of manual data collection forms. The computerised system still needs to be pilot tested and reliability tested. Where a project uses a pre-existing system, the system devel- opment activities are not needed. However, the system must still be reliability tested to ensure that the particular student can use the system objectively.
Sometimes the entire purpose of a study is to do a multiple match reliability study of a system. The research process follows the same stages as those already described as there will still be a purpose to the study and results to be produced. The main difference is that the reliability study is not an additional step preceding the main study, but the two are the same.
A mistake made by some students is that they use commercial sports analysis systems when they are not required for the research. These students are under the impression that if they use these systems they will be awarded a higher mark for their dissertation. There are some projects where the use of such systems is warranted, but where students use them inappropriately they can actually lose marks. These systems require the development of code sets as well as the capturing, storing and tagging of video footage of the performances. This takes time and delays the production of data and results and can mean that the student is not able to analyse as many matches as they could have if other methods had been used. Therefore, students should always remind themselves of the purpose of their study and consider the most effi cient and reliable way of gathering the necessary data to answer the research question. Any data of any type that is not required to produce the results to answer the research question are not relevant and their gather- ing and processing are a distraction.
Some research projects evaluate performance analysis support to a squad of athletes (Jenkins et al., 2007, Kingston, 2009, Martin et al., 2004). The activities involved in such a project will be slightly different to those of the traditional notational analysis study. Projects evaluating the effectiveness of performance analysis support involve the following stages:
1. The initial stage should still consider the purpose of the study and the eventual results that can be produced from it. In sketching the intended results format, there may be a section of qualitative results to be
included, which means that the volume of tables and charts for the quantitative results will be lower than in a traditional notational analy- sis project.
2. The raw data that are needed to produce the information required should be identifi ed.
3. The student needs to have negotiated providing performance analysis support to the squad before any system development effort commences.
The squad need to be aware of what is involved and how it will impact on their preparation for matches.
4. The system development process should involve the coach and the evo- lution of the system should be documented so that this activity can be evaluated later during the research. Once the system is completed, it should be tested for reliability the same as if it was being used in a traditional notational analysis study.
5. Once the fi nal system can be used reliably, the student should consider the main study of the research project. The system will be used in a cycle of activity that may include some or all of the following:
a) recording of the performance;
b) analysing the performance;
c) providing quantitative results to the coach and related video sequences;
d) recording the coach’s decisions about training priorities based on the analysis;
e) providing video and possibly statistical feedback to the athletes;
f) monitoring training sessions before the next match; and
g) recording any tactical or selection decisions made for the next match as a result of the analysis.
6. The squad performance data and the fi eld note observations about the use of the performance analysis support during the match-to-match cycles are analysed and the results are written up.
In planning the project, any activity or task that takes time must be identi- fi ed and scheduled. The number of performances to be analysed needs to be determined and particular performances from which data can be gathered need to be identifi ed. Periods of the academic year where the student has examinations to do or coursework to be submitted should be located on a timetable for the year so that research project activity can be scaled down at these times but emphasised at others. Some tasks depend on the completion of other tasks and such temporal relationships between project tasks need to be represented within the project plan. Ultimately, each task should be time- tabled within the project plan along with key deliverables such as a com- pleted system, reliability results and fi ndings for the main study. The remainder of this chapter describes the detail of activity involved in formu- lating the methods.