be removed from the system and any additional system output needed to fully answer the research question should be added to the system.
Having identifi ed the information to be output by the system and rea- soned how this information will be used to produce the results, the student should identify the raw data that are required to produce these outputs. The details associated with each event to be recorded should be considered.
Hughes and Franks (2004c) listed position on the playing surface, player, action, outcome and time as key data to be recorded for events within fi eld games. The system must be developed to allow effi cient gathering and processing of data and production of information. The student needs to consider if tallying event frequencies is suffi cient or if a chronological sequence of events is required. Where the latter is used, it is necessary to include a separate step after data gathering to process the chronological event list to produce the information required. Where temporal analysis of the events is necessary, tallies lose vital data about the order of events. The system may have to process the data in stages; for example McCorry et al.
(1996) used a set of forms to tally raw frequencies of events in rugby union matches and then used a form that summarised the frequencies from four forms used for each team in each half of a game.
Operational defi nitions
Any variables that have not already been defi ned need to be defi ned during system development. This is required for system outputs as well as the data being entered. Chapters 4 and 7 cover the objective defi nition of variables.
It is worth noting that in performance analysis it is not always possible to be completely objective and very often the quantitative analysis eventually done is an analysis of counts and timings of subjective observer judgements.
Consider the defi nitions of technical effectiveness used by Hughes and Probert (2006) shown in Table 6.1. Terms such as ‘slight pressure’, ‘good technique’ and so on are undefi ned and classifi cation of technical effective- ness is done subjectively. A further issue with this rating system is that two different variables (pressure and quality of technique) have been merged in a restrictive way. For example, excellent technique could be performed under no pressure.
Students should defi ne terms as precisely as possible but should not claim to have made unambiguous defi nitions where they have not done so. It should also be said that assessors should not penalise students for failing to produce precise and unambiguous operational defi nitions where this is clearly an impossible task.
Development issues for manual notation systems
When developing a manual notation system, we should consider the even- tual results to be produced and how the system can provide information to
make the production of the results as effi cient as possible. Consider a system to analyse serving effectiveness in tennis where we wish to be able to deter- mine the percentage of points won when:
serving to the deuce and advantage courts;
•
serving to the left, middle and right of the target serving courts;
•
using kick serves, slice serves and fl at serves;
•
serving on fi rst and second serve;
•
playing different sets;
•
any combination of these fi ve factors.
•
The student needs to design a form that allows data to be analysed according to the above requirements. The use of tallies allows a near immediate inspec- tion of the results and the requirements in this example do not include any temporal patterns, so the student should avoid using a chronological sequence of points if at all possible. Figure 6.1 shows an example of a form that could be used to gather the data for the project in this example. A copy of this form would be used for each set for each player participating in the match under observation. The use of separate forms for each set prevents individual set information being concealed by overall match totals. Overall match totals can be determined by adding the totals in the individual sets. The form contains 72 different tallies that are accumulated during match observation. However, there are areas of the form where information on different serve types, serving to different courts, serving to different areas of the target service court and fi rst or second service can be summed reasonably easily. The easiest way to compute the percentage of points won in different situations is to key the totals in data entry cells into a spreadsheet that has been programmed to cal- culate the percentages of interest. When one considers that there are fi ve dif- ferent factors hypothesised to have an infl uence on the percentage of points won as well as 10 pairs of factors, 10 triplets of factors, fi ve sets of four factors and one set of all fi ve factors that could potentially be considered, the user needs to determine which combinations of factors they are most
Table 6.1 Technical effectiveness ratings for soccer (Hughes and Probert, 2006) Rating Operational Defi nition
+3 Excellent technique performed under pressure +2 Very good technique performed under slight pressure +1 Good technique performed under no pressure
0 Average, standard technique
-1 Poor technique performed under pressure
-2 Very poor technique performed under slight pressure -3 Unacceptable technique performed under no pressure
interested in determining results for. This is essential to focus the research project and prevent the results from becoming unmanageable.
The student could have created a court diagram to allow the location of services to be plotted. This might be useful in some projects, but in the example described here, we only needed to distinguish between the left, middle and right of the service court. The student should always be looking to make the system as simple as possible (Hughes and Franks, 2004c) and should not be entering data that are not going to be used.
Consider the boxing example described by Hughes and Franks (2004d).
This system used shorthand symbols to represent different types of action performed by the contestants. However, the results that Hughes and Franks showed for this match were two pie charts showing the distribution of punches made by each contestant and the following four tables:
1. Total punches thrown by each contestant in each round.
2. The frequency (and percentage) of different types of punch thrown by the two contestants.
3. The number of punches thrown by each contestant when holding.
4. The number of one particular type of punch (the jab) thrown by each contestant in each round.
Instead of having to learn a shorthand notation symbol for each punch type, tallies could have been made in a form like the one shown in Figure 6.2.
This would allow a more effi cient analysis of the recorded data than the system described by Hughes and Franks. The system they described does admittedly support temporal analysis of punch sequences, which is undeni- ably important in martial arts, but such temporal analysis was not required for the study they described.
Serve Type FIRST SERVE
SECOND SERVE Outcome
Slice Kick Flat
Slice Kick Flat
Win Lose Win Lose Win Lose Win Lose Win Lose Win Lose
Deuce
Left Middle Right Left Middle Right
Advantage
Figure 6.1 Manual form for collecting tennis service data
Development issues for computerised systems
Computerised systems can be developed by those with programming skills;
for example the author developed a computerised notational analysis system for tennis using the Modula2 programming language (O’Donoghue and Ingram, 2001). Today, fourth generation languages exist and database pack- ages allow data entry forms to be developed as graphical user interfaces. The main benefi t of this is that the chronological sequence of events can be stored without any loss of temporal information and database querying provides a fl exible and effi cient way of determining cross-tabulated frequencies. The time-consuming process of producing totals from manual event records that are collected in chronological order is an almost instantaneous process when using a computerised system. It is, therefore, recommended that all compu- terised sports analysis systems should store details of every individual event entered rather than keeping running totals. The amount of storage required for this quantitative information is trivial by the standards of today’s compu- ter hard disks and secondary storage devices. For example, the details of all
Figure 6.2 Tally system for boxing analysis
1 2 3 4 5
BOXER:
ROUND Holding Jab Uppercut Hook Body Miss
Not Holding Jab
Uppercut Hook Body Miss BOXER:
ROUND Holding Jab Uppercut Hook Body Miss
Not Holding Jab
Uppercut Hook Body Miss
36,596 tennis points recorded during O’Donoghue and Ingram’s (2001) Grand Slam tennis study can be stored in 7.6 Mbytes of disk space, while the timed sequence of up to 2,000 locomotive movement instances for each of 24 soccer players in O’Donoghue’s (1998) soccer study can be stored in 1.4 Mbytes of disk space. Even though the temporal patterns were not being used in these studies, the fact that the systems can effi ciently provide the necessary outputs means that there are no disadvantages to storing the data as a chron- ological sequence of events. There are, however, numerous advantages to storing the data in this form, including:
Reliability studies can identify all disagreements between independent
•
observations of the same performance. If we merely use the total fre- quencies recorded for events, there is a danger that some errors might be cancelled out by others, giving an impression that the level of agree- ment is better than it actually is. This is illustrated in Chapter 7.
There are potential follow-up studies that can be done if the complete
•
chronological list of events is stored. For example, follow-up studies using O’Donoghue and Ingram’s (2001) Grand Slam tennis data included investigations of temporal patterns in tennis points (Wright and O’Donoghue, 1999) and score line effects (O’Donoghue, 2001a, 2003, Scully and O’Donoghue, 1999).
If video recordings of the match become available, the data can be
•
transformed into a format that can be used to demonstrate integrated match data and video systems.
The computerised data collection system for tennis developed by O’Donoghue and Ingram (2001) is used as an example of a computerised system. This system required some initial match identifi cation details to be entered by the operator before the entry of individual point data commenced. The match identifi cation details included:
the gender of the players;
•
the tournament being played;
•
the players contesting the match;
•
the player who served fi rst in the match section;
•
the score (sets, games and points) at the start of the match section.
•
The gender was necessary to allow the system to determine whether the match was the best of three or fi ve sets and the tournament was necessary to allow the system to determine whether or not a tiebreaker was played in the fi nal set (this is currently the case at the US Open only). The system required the following details of each point to be entered:
whether the point emanated from a fi rst or second service;
•
timing details (inter-serve times, rally-time and inter-point time);
•
number of shots played in the rally;
•
type of point (ace, double fault, serve winner, return winner, baseline
•
rally or net point);
outcome of point (winning player and whether point was won with a
•
winner or an opponent error); this included the outcome of attacking the net;
the cause of a player attacking the net.
•
The timing details were entered using the function keys F1, F2 and F3. F1 represents a fi rst service, F2 a second service and F3 the end of the point.
The operator of the system pressed these keys while watching the point. The use of function keys to time rallies and other timings also allows the system to determine if a point emanated from a fi rst or second serve. The type of point was derived from the classifi cation of points shown in Figure 6.3, which also identifi es the various winning and losing outcomes. This was mapped onto the menu structure implemented within the system.
With players approaching the net, the point classifi cation scheme also addressed the cause and outcome of players approaching the net. The causes of players approaching the net were classifi ed into three types. The fi rst of these is where the player attacks the net to pressurise the opponent. This includes following up a good service, following up a good approach shot
Points / Outcomes
Ace Double Fault Serve Winner Return Winner Server to Net Receiver to Net Baseline Rally
Server wins at Net Server Retreats Server loses at Net
Volley/Smash Drop/Drive Opp Error
Volley/Smash Drop/Drive Opp Error Server wins Receiver wins
Server wins Receiver wins Receiver wins Server wins Passed Lobbed Net Error
Passed Lobbed Net Error Receiver wins at Net
Receiver Retreats
Receiver loses at Net Winner
Error Winner Error Winner Error Winner Error
Winner Error Winner Error Figure 6.3 Classifi cation of point types in tennis
and ‘chipping and charging’ to attack an opponent’s service. The second cause of approaching the net is where the player is drawn to the net by an opponent’s drop shot or short-length shot. The third cause of approaching the net is luck, where neither the player nor the opponent had decided that the player should approach the net. This may occur as the result of the ball being defl ected or slowed by the net, or as the result of a miss-hit. The out- comes of approaching the net were classifi ed as follows:
points won at the net using volleys, overhead shots/smashes, drive
•
winners, drop shots and by opponent errors;
points lost at the net by being lobbed, passed or making an error at the
• net;
retreating from the net, even if the player or opponent eventually returns
•
to the net; only details of the fi rst approach to the net within a point are included.
Computerised systems have the ability to automatically generate values that may be of use in further analyses of the data. As a rule, a computerised system should never require an operator to enter anything that can be com- puted automatically. O’Donoghue and Ingram’s (2001) system automati- cally updated the score and, where necessary, the serving player. The system also allowed non-scoring points, such as ‘lets’ to be identifi ed. If a penalty point was awarded, the system allowed the score at the beginning of the next point to be altered accordingly.
The order in which data about an event is entered should map the opera- tor’s mental model of the event. This mental model is a sequence of event details that is built up by the observer while observing the event. For example, O’Donoghue and Ingram’s (2001) system was developed to record point information in the following sequence:
1. the number of shots played in the point;
2. the point type;
3. the cause of going to the net if the point was a net point;
4. the outcome of the point if it was a baseline or net point.
This matched the mental sequence of information that the observer built up while watching the point. A net point ending with a successful volley would be observed with the operator thinking, ‘server to net – following a good approach shot – volley winner’. An early version of the system was almost unusable because it required the user to enter the outcome of the net point and then the cause. Simply changing the order so that the cause of approach- ing the net was entered before the outcome of the net point made the system straightforward to use because the data entry process matched the opera- tor’s mental model of the point.
The operator had to enter the point details during the 20s interval that followed the point. Therefore, the system interface had to be designed in a way to make this as easy as possible for the user. A series of menus were presented, with the user required to type in a numerical code representing the particular option chosen in each case. Using the keyboard was much more effi cient than using a graphical user interface. The system updated the score and when the operator saw that the score on the system matched the score called by the umpire at the end of the point, the point data record would be confi rmed. If there was any discrepancy here, the operator had to re-enter the point details correctly before the next point started.
The advantages of computerised systems are that they can provide effi - cient selective searching of large databases of match data. This applies to both scientifi c investigations as well as in coaching contexts. O’Donoghue and Ingram’s (2001) tennis system had a multiple match analysis facility that produced a spreadsheet of variables (columns) by cases (matches) for all 252 matches in their study, which was input into a statistical analysis package. This avoided laborious data transcribing, which would have included errors.
Development issues for systems using commercial packages
There are advantages and disadvantages to using the commercial sports analysis packages that are available today. The main disadvantage is that these systems are designed as general purpose integrated video and match database packages that can be tailored for use with a sport of the user’s choice. Such systems cannot be all things to all people; for example, the automatic updating of score line and server in O’Donoghue and Ingram’s (2001) special purpose tennis analysis system could not easily be included in a system developed using one of the commercial packages. Another disad- vantage is that these systems are used with digital video, which require storage space. A student should not use a commercial video analysis package unless the video component of the system is essential to the research.
Capturing video, compressing and then tagging it all require time that delays the production of the main system outputs. This, in turn, can reduce the number of matches that can be analysed within the research project. The projects where the use of such systems is essential include the following:
Analysis of coach behaviour requires repeated viewing of behaviours in
•
order to classify them (Donnelly and O’Donoghue, 2008).
Analysis of detailed movements performed by players, as in the
•
Bloomfi eld Movement Classifi cation (Bloomfi eld et al., 2004).
Studying the effectiveness of video analysis (Jenkins
• et al., 2007).
Biomechanical analysis of technique where using computerised pack-
•
ages is essential.
Detailed timing of events in athletics such as touchdown times in hurdles
•
events (Greene et al., 2008).
There are advantages to using commercial video analysis systems, for example such systems may be essential if they are for some of the examples listed above.
Another advantage of these systems is that they do not require the student to be able to develop computer software. The use of such packages is also a good skill for the student to have on their curriculum vitae (résumé).
Piloting
No matter whether the system to be used is a manual notation system, a special purpose computerised system or a system implemented using a com- mercial sports analysis package, the system should still undergo a pilot study to ensure that it can be used for data collection as intended. Pilot studies allow any unforeseen problems to be experienced and rectifi ed before data collection commences. Systems may undergo a series of refi nements during pilot testing, with several versions of the system being tested before the fi nal system is agreed (O’Donoghue and Longville, 2004). With word limits imposed on dissertations, it is not possible to outline all of the details of all pilot versions of the system. Indeed, to make the study replicable, only the fi nal version of the system has to be described to the reader. However, it is worth describing the types of refi nement made to the system during pilot studies and the student should record this information during system devel- opment so that it can be used when writing up the methods chapter.
Description of the fi nal system
The fi nal system must be described in detail, using appendices where neces- sary. The criteria for recording an event should be outlined and the defi ni- tion of data variables should be provided. The data collection forms or the interface of the computerised system used should be illustrated, using appen- dices if necessary. An important point to raise here is that the methods chapter should not read like a user manual. It is not necessary to describe every click of the left or right button of the mouse device when accessing menus or on-screen buttons of a graphical user interface.
In addition to describing the manual or computerised notational analysis system, it is also necessary to describe the observation process and opera- tion of the system during data collection. This is because the operator is part of the system and the success of the system largely depends on the user’s ability to observe the necessary detail during sports performances. For example, O’Donoghue and Ingram (2001) described how the trained observer should watch a tennis match to be able to record the cause of a player approaching the net. Tennis spectators habitually watch a player play a shot and then immediately focus on the opponent until the opponent