Validity can be described as the ability of the measuring device to indicate what it is designed to measure and reliability is described as the repeatability of measurements (i.e., the absence of measurement error) (Atkinson and Nevill, 1998). The validity of a measuring device also depends on its reliability (O’Donoghue, 2010). Reliability tests are used to evaluate the consistency of measurements (test-retest reliability) (O’Donoghue, 2010) in both intra-devices (comparing the measuring instrument with itself) and inter-devices (comparing two equal instruments). It is important to assess reliability to ensure that these new analysis methods are sensitive enough to detect any changes in player performance (Atkinson and Nevill, 1998).
In high sports performance, coaches and researchers are constantly searching for new ways to measure players’
performance in order to achieve a competitive advantage (Barris and Button, 2008). However, quantifying performance can be extremely difficult to measure directly (Carling et al., 2009), so that coaches and sports scientists have investigated indirect measurement methods (for example, accelerometers) with high accuracy compared with established techniques known as “gold standard” or “criterion measure” (Bland and Altman, 1999).
If the accelerometer is reliable, the same value will be measured each time that the same movement is performed (considering the same conditions and procedures). However, if the accelerometer is unreliable, the measured value may vary from session to session and the measurement error would be above what is considered acceptable (Atkinson and Nevill, 1998). Also, if an accelerometer is to be considered valid and reliable, it has to measure the same value as well as to measure the correct value every time. For example, if an athlete runs at 10 km/h on a treadmill five times and the maximum value of accelerations for each step is 5 g, a valid and reliable accelerometer must measure 5 g in all trials. However, if the accelerometer measures 10 g every time it is reliable but not valid.
Alternatively, if the accelerometer measures five different values (for example, 2 g, 4 g, 6 g, 8 g and 10 g) it is not reliable and valid. In the real context, if team staff use non-valid or/and non-reliable accelerometers, players could stop training too early or too late and do not achieve the correct adaptations according to the training plan. This fact could lead to a decrease in the athlete’s performance and an increase in the injury risk. Therefore, it is extremely important to ensure that the accelerometer contained in the tracking device is both valid and reliable.
Table 5.1 shows the investigations that analyze the reliability and validity of the accelerometers that contain the inertial devices used in the monitoring of team sports. Good-to-moderate reliability was found in accelerometers through total accelerations as well as specific accelerometry-based external workload indexes designed by the manufacturers. Instead, the analysis per axis presented worse results, being the z-axis which presented the best results. For validity, studies have examined concurrent (respect to other accelerometers or instruments that detect acceleration) and convergent validity (respect to internal and external workload variables different than acceleration). Both methods have obtained good validity results, except in SPI-ProX II.
Table 5.1 Validity and reliability results published in the scientific literature related to tracking devices with accelerometers Company Device /
components SF Assessment Trials Criterion Variable Error Interp
Catapult MinimaxX 2.0 1 3-D accelerometer
±16g
Kionix KXP94
100 Reliability Static trials Intra-unit AcelT CV=
1.01%
Good
Inter-unit AcelT CV=
1.10% Good
Dynamic trials Intra-unit AcelT CV= 0.91–
1.05%
Good
Inter-unit AcelT CV= 1.02–
1.04%
Good
Sport-specific Inter-unit AcelT CV=
1.94%
Good MinimaxX S4
1 3-D accelerometer
±16g
Kionix KXP94
Reliability Incremental
treadmill Inter-unit PL
PLx PLy PLz
CV= 5.2–
5.9%
CV=
7.5–
9.1%
CV=
6.3–
7.3%
CV=
11.4–
12.0%
Moder Mod Mod Poo
Validity Incremental
treadmill HR PL r = 0.93–
0.98 Good
VO2max PL r = 0.94-
0.98
Good Optimeye S5
3 3-D accelerometers
±16g Model not mentioned
100 Reliability Dynamic trials Intra-unit x-axis Peak AcelT Device PL Calculated PL
CV= 0.07–
4.65%
CV=
0.02–
4.77%
CV=
0.04–
4.73%
Good Goo Goo
y-axis AcelT Device PL Calculated PL
CV= 0.02–
5.00%
CV=
0.02–
5.20%
CV=
0.04–
5.38%
Good Goo mod Goo mod
z-axis Peak AcelT Device PL Calculated PL
CV= 0.03–
0.24%
CV=
0.03- 0.10%
CV=
0.10–
0.73%
Good Goo Goo
Company Device / components
SF Assessment Trials Criterion Variable Error Interp
Inter-unit Device PL
x-axis y-axis z-axis AcelT x-axis y-axis z-axis
CV=
75.86–
245.28%
CV=
100.69–
157.66%
CV=
69.20–
106.33%
CV=
67.54–
190.72%
CV=
71.08–
93.43%
CV=
70.68–
279.23%
Poor Poo Poo Poo Poo Poo
Validity Dynamic trials Accelerometer J353B31, PCB Piezoelectronics
AcelT x-axis y-axis z-axis
%diff= 2.4–
22.3%
%diff= 1.8–
23.5%
%diff= 1.0–
4.9%
Good–
Goo poo Goo
Validity Walking, running, jumping and sport-specific movements
VICON 200 Hz
Mean AcelT Peak AcelT
CV= 2.6–
3.8%
CV=
4.9–
7.1%
Good Mod
Reliability Sport-specific movement in laboratory
Inter-unit PL
IMA
CV= 2.9–
7%
Good- mod Training
sessions
Inter-unit PL
IMA
CV= 0.8–
1.1%
CV=
1.8–
5.3%
Good Goo mod
GPSports SPI-ProX II 1 3-D accelerometer
±8g Bosch BMA150
100 Reliability Dynamic trials Intra-unit AcelT CV= 1.87–
2.21%
Good
Validity Static testing Accelerometer ADXL345 ±8g 100 Hz
AcelT CV= 27.5–
30.5% Poor Mediatronic Zephyr
Bioharness 3-D
accelerometer Model and number not mentioned
100 Reliability Displacements at different speeds
Inter-unit AcelT CV= 10.3–
12.6% Poor
Company Device / components
SF Assessment Trials Criterion Variable Error Interp
Test-retest AcelT CV= 13.2–
15.8% Poor RealTrack
Systems
WIMU PRO 4 3-D accelerometers
±16, ±16, ±32
& ±400g Model not mentioned
100 Reliability Static trials Intra-unit AcelT CV= 0.23–
0.40%
Good
Inter-unit AcelT CV= 0.38–
0.78%
Good
Dynamic trials Intra-unit AcelT CV= 0.55–
0.12%
Good
Inter-unit AcelT CV= 0.59–
0.66%
Good Incremental test Inter-unit AcelT CV= 2.05–
2.52% Good
SAFT90 Inter-unit AcelT CV= 2.46–
3.20%
Good
Reliability Treadmill Test-retest PLRT ICC=
0.93–
0.99 CV=
3.52–
5.37%
Good
Athletic track Test-retest PLRT ICC=
0.88–
0.96 CV=
6.54–
8.12%
Moder
Validity Treadmill HR
SmO2 PLRT
PLRT
r= 0.99 r= 0.77–
0.79
Good Mod Athletic track HR
SmO2 PLRT
PLRT
r= 0.99 r= 0.59–
0.61
Good Poo Note: CV: Coefficient of variation; %diff: Percentage of difference; ICC: Intraclass correlation coefficient; r: Pearson correlation coefficient; AcelT: Total acceleration; PL: PlayerLoad by Catapult; PLRT: Player Load by RealTrack Systems; HR: Heart rate, SmO2: muscle oxygen saturation; VO2max: Maximal oxygen consumption; SAFT90: specific aerobic football test.