VALIDATION AND RELIABILITY BETWEEN EXTERNAL LOAD ANALYSIS DEVICES FOR SOCCER PLAYERS

Objective: To test the reliability between two instruments with different analysis mechanisms, either by GPS (model GPSPORTS ® ) or by video analysis (InStat For Players ® ), relating the results of total distance covered and distance at high speed ≥ 20km/h (Very High-Intensity Running Distance, VHIR) during official soccer matches. Study Design: This is a methodological study. Data from 35 male professional soccer athletes from all tactical positions were included. Age 29.2 (± 4.8 years) and body fat 9.9 (± 1.7%), excluding goalkeepers (102 individual analyzes) were collected in official matches. In the data analysis, descriptive statistics procedures were used to characterize the sample and the intraclass correlation coefficient (ICC) was used to verify the agreement on the stability and internal consistency of the tests with 95% confidence intervals (CI). Results: The ICC in the case of the total distance traveled variable was significant 0,914 (0,876; 0,941) and indicated a very high agreement, with the linear correlation coefficient indicating a strong positive correlation (p <0.001). The ICC for the VHIR variable was not significant, although the linear correlation coefficient indicates a strong positive correlation (p <0.001). Clinical Relevance Statment: This study reveals that there is good agreement in the comparison of two systems designed to analyze the movement demands of each professional soccer athlete in relation to the total distance covered. Level of Evidence I; Methodological Study - Investigation of a diagnostic test.


INTRODUCTION
Quantifying the external load of athletes is performed to reduce the risk of injury and to improve performance. 1,2 Tactical performance in a soccer match depends on the quality of the athletes' actions in space and time during the game in order to be successful. 3 Due to the multifactorial requirements for success in soccer, many attempts have been made to elucidate the physiological demands of a soccer match based on estimates of distance traveled and fluctuations in running intensity during a match. 2,4 Technological innovations have led to new possibilities to capture accurate spatiotemporal information from all athletes and to unveil the dynamics and complexity of soccer games. The capture and analysis of this data is of paramount importance to elaborate the technical strategy of each athlete and the team as a whole, in addition to carrying out load control through generating and analyzing post-game and post-training data. 1,5,6 Such data in professional soccer can be captured using the Global Position System (GPS) and also by semi-automatic video analysis devices, both of which are responsible for generating positioning data, speeds and monitoring load demand. 7,8 They enable evaluating the most important physical actions performed by the athletes, which are the total distance covered, the number of changes of direction and the distance covered at high speed which means ≥ 20km/h (Very High-Intensity Runnig Distance, VHIR). 9 Although the use of electronic tracking devices is allowed, some studies have reported an intermittent loss of signal when downloading the raw data from the GPS device due to the fact that the engineering of some soccer stadiums ends up blocking the signal transmission, interfering in the data acquisition. 2,10,11, Thus, the use of video technology in official games was adhered to and being used together with GPS when possible, since the multi-camera technology does not suffer interference related to the environment structure, thus ensuring successful data acquisition. 12,13 Some researchers report a lack of studies which compare the results obtained by the two methods, in addition to showing that several semiautomatic camera systems tend to report slightly to moderately greater distances covered in medium and high intensity than GPS technology. 14 Thus, the objective of the present study was to compare the variables of total distance traveled and VHIR during professional soccer matches in different stadiums and arenas, collected simultaneously from the video system (InStat For Player) with the same data obtained from a GPS device (GPSports®, model SPI HPU), and then to evaluate the agreement between them. All data in this study were collected during matches in official competitions and under real use conditions of the devices in the practice of professional soccer.

MATERIALS AND METHODS
This is a methodological study and it was described according to the Guidelines for Reporting Reliability and Agreement Studies -GRRAS 15 and Consensus-based Standards for the Selection of Health Status Measurement Instruments -COSMIN 16 initiative in order to improve the methodological quality of the study. The study was preceded by the approval of the Research Ethics Committee of the Faculdade de Ciências Médicas de Minas Gerais -15737819. 6.0000.5134. The performance data of 35 athletes from a Brazilian professional soccer team excluding goalkeepers from a total of 102 individual analyzes were collected in official championship games during the 2019 season.
Athletes from all tactical positions were included: central defenders, lateral defenders, central and lateral midfielders and forwards. The athletes belonged to the squad of a Brazilian first division soccer team and had professional experience of at least two years in training and competition at national and international level, recognized by the Brazilian Football Confederation (CBF) and South American Football Confederation (CONMEBOL).
The sample size was calculated to test the significance of the CCI. 17 The significance of 5%, the minimum power of 80%, to test the significance of a CCI of 0.5 inoculation for at least 28 athletes. The athletes were informed about the objectives of the study and the confidentiality of their identity, and all signed the Free and Informed Consent Form.
A GPS unit, model GPSports ® brand GPS, SPI HPU model, was attached to each of the athletes individually before each one entered the field for an official soccer match. 7 Semi-automatic video analysis devices such as InStat ® also achieve sports performance analysis in the same way as GPS, providing a professional tool for individual performance evaluation (Instat For Players). This technology is an external load analysis methodology based on multiple high definition cameras which accompany athletes placed around the soccer field. This system reproduces the trajectories of athletes around the field throughout the matche, and allows researchers with access to the trajectory data to study the athletes' movements, speed, acceleration, total distance covered, VHIR and the interactions between them. 8 The GPS data sets were adjusted according to the InStat data at the beginning of each time; however, data from some matches was eliminated because a signal loss was detected by downloading the raw data of all possible analyzes from the GPS device, making its use unfeasible. We used data from six teams of professional soccer games in three matches, in which the athletes played with the GPS device and were also analyzed by the InStat®. These performance data were separately recorded from each game time in order to enable data comparison from the two systems and avoid evaluation errors, specifically using the variables total distance traveled and VHIR.
Data were obtained from 35 athletes in relation to the total distance covered and VHIR in matches situations (for some, first and second time) for a total of 102 measurements in order to assess the internal consistency of the measurements and the reproducibility of the measurements made by the GPS and InStat systems. The averages of each of the measures were obtained for each system, the average differences between the two systems, ICC, and Pearson's linear correlation coefficient (r), with all of these measures being implemented with their respective 95% confidence intervals. The magnitude of the ICC was analyzed according to the criterion described by Pons: 2 trivial (≤0.1), small (0.11 to 0.30), moderate (0.31 to 0.5), large (0.51 to 0.7) and very large (0.71 to 0.9), and almost perfect (0.91 to 1).
The normality of the measurement differences was assessed using the Shapiro-Wilk test. The Student's t-test for paired samples was used to compare the mean differences, and the significance of the ICC and r were also tested. The analysis was developed using the free R version 3.5.3 software program and p<0.05 was considered significant. Mean differences between the two systems, mean differences between the two systems, intraclass correlation coefficients (ICC), Pearson linear correlation coefficient (r), were made available as means for each of the systems, all of these measures with their interval intervals. 95% confidence. The magnitude of the ICC was analyzed according to the criterion described by Pons. 2 The normality of the differences in the measurements was assessed using the Shapiro-Wilk test. To compare the mean differences, the Student's t-test was used for the paired tests, and the significance of the ICC and the r were also tested. Simple linear regression models were built to describe the relationship between the measurements made by GPS and InStat. Scatter plots, regression equation and determination coefficient were produced. The quality of the adjustment was verified by analyzing the residues, where normality, homoscedasticity and presence of outliers were evaluated. The Bland-Altman graph was constructed to verify that the difference between the measures was reasonable to assume an agreement between them. The analysis was performed using the free software R version 4.0.0 and p <0.05 was considered significant.
For the variables total distance covered and VHIR there was a significant difference in the measurements obtained via GPS and via InStat (p = 0.021 for distance and p <0.001 for VHIR). In the case of the distance variable, the ICC was significant and indicated almost perfect agreement and the linear correlation coefficient indicated a strong positive correlation (p <0.001). For the VHIR variable, the ICC was significant and indicated moderate agreement, and the linear correlation coefficient indicated a strong positive correlation (p <0.001) ( Table 1).
The simple linear regression models for the distance and distance VHIR variables showed good adjustments, with normal, homocedastic residues and without outliers. The values obtained by the GPS can be used to predict the InStat values, using the equations presented in Figures 1a and 1b. Most of the differences in distance measurements made by GPS and InStat were between -402 (8% of the average between the two distances) and 318 (6% of the average between the two distances), being considered a reasonable variation (Figure 2a). In relation to the VHIR measures, most measures were between -270 (79.6% of the average between the two VHIR) and 28.99 (8.6% of the average between the two VHIR), in this case, a large variation that shows that the VHIR measured by GPS tend to be much smaller than those measured by InStat (Figure 2b).

DISCUSSION
According to the results obtained in this study, we highlight that InStat moderately overestimated the values of the VHIR compared to the GPS. On the other hand, the variable total distance covered did not show any significant difference between the two technologies. The differences between the two systems were smaller than in previous studies, possibly due to technological advances, 11,18 such as the increase in the number of cameras (in the case of multi-camera technologies) to obtain greater coverage of the stadium from more angles and improvement in the resolution quality of the images, detecting and automatically tracking each player by their identification number. 19 These results are consistent with the findings of the study by Edgecomb and Norton 20 , concluding that both GPS technology and the computer/video tracking system involve systematic errors, overestimating the distance covered. It is noteworthy that the VHIR is one of the measures which may be related to injury prevention. 1,2 There was a significant difference in the measurements obtained via GPS and via InStat (p<0.001) in the total distance covered variable in our study, but the ICC was significant, indicating great agreement and a strong positive linear correlation coefficient. The ICC for the distance variable was 0.88, inferring a good agreement between the GPSports ® SPI HPU model and the InStat For Player, and therefore the combined use of the two instruments can be performed based on these results. Some studies have shown that several semi-automatic camera systems tend to report light to moderately greater distances 4,14 covered at medium and high intensity than GPS technology. The results in these investigations highlighted that all systems similarly detected the external load produced during the start, and there were differences between the  20,21,22 have also evaluated the measurement accuracy of the tracking technologies most used in professional team sports (i.e. semi-automatic video technology with multiple cameras, ultra-wide band -UWB, LPS and GPS), concluding that the differences between the technologies were not as pronounced in distances and speeds, but all technologies resulted in the magnitude of error increasing as the speed of the tracked object also increased. The results revealed technology-dependent variations in precision in the video tracking system. Some GPS units even measured two to six times more acceleration/ deceleration occurrences than other units in these studies. According to the authors, this phenomenon could be explained due to the difference in the software version installed in each global positioning device. 13,22,23 Other studies 9,24 evaluated the convergent validity and test-retest reliability between GPS devices of the same model in which it was observed that there was a difference in the results for the same variable and for the same test, concluding that some GPS models could vary in results between themselves, thus making it essential to carry out reliability studies to guarantee the reproducibility of the results from the equipment. On the other hand, most studies were carried out in non-ecological environments, creating circuits which simulate real conditions of competition 13,20,25 and therefore it is interesting to note that our study compared Instat with real tracking data during official matches using a GPS device. This is due to the recurring signal loss from the engineering of certain soccer stadiums, especially those with metallic covers, as described in the literature, 10,18 making it impossible to use these data from some matches in our study. The problem was overcome by using data from other games of the same season which did not experience this signal loss. We emphasize that the measurements were carried out in different environmental situations in stadiums in different geographical locations at different times, in comparison with other authors who carried out the study in a single stadium and at a specific time. 26 The imperfection in the correlation between the two systems remains a major challenge for analyzing training and game data. The difficulties in obtaining good quality GPS signal capture in games in indoor stadiums reinforced the increasing use of control systems through cameras. At the same time, the use of the camera system in the training camps also presents some difficulties such as the high installation cost and the demand for a longer time than the GPS for the data analysis and to prepare information reports. Thus, standardizing a single system to be used in training and official games still seems to be an important issue to be evaluated in other studies.

CONCLUSION
This study shows that there was a good agreement between the GPS systems and InStat in comparing the two instruments designed to analyze the movement of each player in a professional soccer match with regard to the total distance covered. In relation to the first, there is a tendency to moderately overvalue the variables analyzed in meters, and the statistical analysis indicated a significant ICC and a very large agreement between the two devices. Therefore, these data provide high relevance for analyzing real situations of correspondence related to physical demands. However, with the instruments respect to the VHIR variable were not reliable, with a significant difference in the measurements obtained between both technologies.