SciELO - Scientific Electronic Library Online

 
vol.22 issue4The effects of a global postural exercise program on temporomandibular disorderAnthropometric analysis and performance characteristics to predict selection in young male and female handball players author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

Share


Motriz: Revista de Educação Física

On-line version ISSN 1980-6574

Motriz: rev. educ. fis. vol.22 no.4 Rio Claro Oct./Dec. 2016

https://doi.org/10.1590/s1980-6574201600040010 

Original Article

Creation, validation, and reliability of a shooting simulator instrument for reaction time evaluation

Ellen dos Santos Soares1  * 

Sara Teresinha Corazza1 

Ana Carla Piovesan1 

Renato Preigschadt de Azevedo1 

Silvio José Lemos Vasconcellos1 

1Universidade Federal de Santa Maria, Santa Maria, RS. Brasil


Abstract

"Creation, validation, and reliability of a shooting simulator instrument for reaction time evaluation." The aim of this study was to develop, validate, and verify the reliability of a shooting simulator instrument for reaction time evaluation. 90 Santa Maria Air Base military personnel participated on the study. Software was developed for use with an electronic gun where participants performed two shooting task tests: simple reaction time and choice reaction time. The results of concurrent validity were satisfactory, no significant differences were found between the two instruments and good agreement was observed. The reliability results were significant in both tests. The instrument can be used for research purposes, or for the purposes of military training as a simple, low-cost tool involving speed, accuracy, and decision-making shooting tasks.

Keywords software; reaction time; military

Introduction

Motor reaction time is a measure widely used in research as an indicator of sensorimotor performance by analyzing the time spent by subjects when performing tasks1. Researchers have investigated the reaction time in different populations, such as children 2, adults 3,4, the elderly 5,6, and special populations 7. Reaction Time (RT) is the time interval between the presentation of a stimulus and the beginning of an answer 8. Accordingly, it can be simple reaction time (SRT), when there is a single stimulus to be answered with only one possible answer, or choice reaction time (CRT), when there are two or more stimuli to be answered with the most adequate response among several possible alternatives 8.

In broad terms, the TR varies according to the sex, age, and task complexity. Men in general are faster than women 9, with maximum performance in this variable being reached between 20 and 30 years old and later declining 2.

When the reaction time is extremely fast, conducting research with this variable requires the use of specific software to register the time between the presentation of stimuli and the early movements 3. Over time, different tools have been developed for the RT assessment using different stimuli, especially visual and auditory, in simple and complex tasks, among which manual tasks such as pressing a mouse button or keys of a keyboard on a computer are the most common 10-12.

Manual tasks have been used in different contexts, however, to evaluate the reaction rate in more specific situations, so it is necessary to create tools that enable the reliable execution of the appropriate tasks in the context in which individuals are embedded. Thus, in situations where specific skills are required, such as in occupations involving the use of firearms, it is appropriate that the evaluation is as close as possible to reality. In the case of military professionals, for example, the skill required to achieve shooting efficiency is acquired by means of learning the fundamentals of shooting in activities where directed attention to important events or objects within a scene in real time is required 13,14.

In this regard, it is important that the assessment tool simulate, even in a simplified way, the variable component of reality, and reproduce the real system's main features, which will make the assessment more accurate and reliable. Therefore, the objective of this study was to develop, validate, and establish the reliability of a shooting simulator instrument for the evaluation of reaction time.

Method

The study complied with the standards and criteria required by Law 196/96 of the National Health Council (CNS). The research project was submitted to the Ethics Committee on Human Research of the Federal University of Santa Maria (UFSM) and approved under 16764113.5.0000.5346 protocol. All participants were informed about the research purpose and signed an Informed Consent Form (ICF).

Participants and Procedures

The group was intentionally selected, and was composed of male military personnel (n = 90), between 18 and 50 years old, belonging to Infantry Battalions and Air Police from Santa Maria Air Base in (BASM). The factors adopted as inclusion criteria were as follows: being in the pre-determined age group, accomplishing the same shot tasks related to frequency and type of training, and having not reported diagnoses of disease, or physical or mental limitations that prevented testing.

All military participating in the study underwent the same training developed by BASM every six months. Training activities involved free shot, in which the individual is positioned at a certain distance from a central target with a total of ten weapon shots; and the instant defense shot, in which targets are presented in black, representing the enemy and in white, representing the absence of danger, in which the individual must fire shots only at black targets. The military positioning for these activities can be standing, sitting or squatting.

Assessments occurred on the premises of BASM during working hours. Before starting data collection all participants were briefed and informed about the objectives and procedures adopted in the research, and they completed an interview with identification information. Then an interview about possible variables that could influence the testing, such as sleepiness or quality of sleep at the night before, tiredness, alcohol or caffeine intake, and physical activity practice was performed (Figure 1). After the tests TRS and TRE were performed. These procedures were performed in two assessment days (test and retest), within an interval of 24 hours.

The subjects were divided into three groups (n=30 in each) according to age: G1 (between 18 and 29 years old); G2 (between 30 and 39 years old) and G3 (between 40 and 50 years old).

Figure 1 Sample characterization 

Instrument construction

The reaction time in shooting task test (RTST) consisted of specific software using an RF Top Gun electronic wireless pistol compatible with Windows Vista (32bit), XP (32bit), and all types of screen monitors (CRT/LCD/Plasma/Projectors). The software, built on the Java SE 7 platform, regardless of a computer platform, is composed of two tasks (SRT and CRT) using images that are projected on a computer screen as stimuli in the individual evaluation. The main tasks are to shoot (pulling the trigger) as quickly as possible and select the most appropriate response according to the presented stimulus. For the elaboration of the images used in the software, actors were hired and image rights assigned by contract. In to the test, the stimulus is represented by the image of a subject characterized as a criminal, dressed in black, or the image of a victim, dressed in white (Figure 2). The stimuli are presented twenty times for each task, randomly, with a minimum time of 100 milliseconds.

In order to accomplish the tests, a closed room is needed so that the lighting conditions will not interfere with the action of the gun. The test begins with the individual in a shooting position (both arms forward and gripping the gun with both hands), with the gun pointing toward the target located on the computer screen (Figure 2). Minimum body movement is required, since the reaction time does not involve performing movements, but only the onset of response.

Figure 2 Positioning for the test realization and representation of images used as stimuli in the tasks of SRT and CRT. 

At the end of each task the software generates the mean time, in milliseconds, from the median 10 attempts (excluding the 5 best and worst times) and further, the number of errors in advance or shots off target, as well as response errors. After calculating the mean value, a time of 84 milliseconds is deducted from mean value due to the transmission error between the output timing of the stimulus of the gun and the arrival of the response on the computer screen, considering the wireless condition of the electronic gun.

Both tasks are described below:

Task 1: Simple reaction time

In this task only the gun trigger is used, being pulled with the index finger of the hand of choice. A single stimulus appears on the screen represented by the image of a criminal, so that, when viewing the stimulus, the individual should pull the trigger as fast as possible. The SRT mean and the number of anticipation errors is generated.

Task 2: Choice reaction time

In this task two buttons are used (the trigger and a button below the trigger of the gun). The image of the criminal or the victim was randomly presented on the screen. When viewing the image of the criminal, the subject should pull the trigger, whereas when seeing the image of the victim, the subject must press the button that is below the trigger of the gun. To develop this task the subject is instructed to place the index finger of one hand (of his choice) on the trigger and the other hand on the button below the trigger. The software calculates the CRT and generates the number of response errors and errors in advance or shots off target.

Statistical Analysis

Data were analyzed using SPSS version 14.0 for Windows with significance level of 5%. For a description of the sample, we used descriptive statistics (mean, standard deviation, and percentage frequency). The normality of variables was verified by the Shapiro-Wilk test. To analyze the validity of the instrument a t test was used for paired samples and the Bland-Altman diagrams for the application of the gold standard method 15. We used the intraclass correlation coefficient to test the reliability of the instrument, considering ICC > 0.70 as appropriate values 16.

Analysis of Concurrent Validity and Reliability

To establish the validity of RTST the gold standard method was used 15. The level of concurrent validity was determined by comparing the measurements obtained in the reaction time test that were created and validated in a previous study, and used as a benchmark to measure the proposed new instrument 11. A paired samples t test was initially performed to compare the mean values between the two instruments, and finally the Bland-Altman diagrams were produced for the analysis of agreement between the two measurements 16.

The reliability was investigated by means of temporal measures stability (reproducibility test, and retest). Testing for both tasks SRT and CRT were performed in two days, with an interval of 24 hours between each evaluation. For the test and retest procedures the environmental conditions were controlled, with both assessments done in the morning at the same place, a closed room with a temperature of 28 ° C.

To analyze the reproducibility of the instrument the intraclass correlation coefficient (ICC) was used based on the parameters of the repeated measures ANOVA 16, as shown below:

in which MSs = mean square for subjects

MSe = mean square for error

Results

The 90 subjects were divided into three groups for analyzation (n=30 in each group) according to age: G1 (between 18 and 29 years old); G2 (between 30 and 39 years old) and G3 (between 40 and 50 years old). The data distribution analysis using the Shapiro-Wilk test showed normality in all groups (p> .05).

Regarding concurrent validity, as shown in Table 1, for the three analyzed groups, no significant differences (p <.05) between the two tests in the tasks of SRT and CRT were found.

Table 1 Analysis of concurrent validity (t test for paired samples) and descriptive figures for the SRT and CRT (milliseconds) for the reference test and the RTST 

Groups Test Minimum - Maximum Mean ± DP t p
G1 SRT1 229 - 408 310.63 ± 40.544 .300 .766
SRT2 233 - 368 308.03 ± 33.627
CRT1 331 - 637 491.70 ± 78.488 1.209 .236
CRT2 404 - 554 474.97 ± 41.001
G2 SRT1 219 - 370 306.00 ± 38.005 .188 .853
SRT2 204 - 405 304.27 ± 46.334
CRT1 411 - 589 511.57 ± 48.975 .689 .496
CRT2 426 - 599 507.00 ± 45.626
G3 SRT1 212 - 473 336.40 ± 57.146 1.659 .108
SRT2 265 - 447 315.73 ± 43.831
CRT1 437 - 651 541.33 ± 60.439 1.783 .085
CRT2 428 - 629 526.03 ± 54.546

Note1: reference test and 2: RTST; SD: standard deviation.

The results of the analysis of agreement between subjects for the three groups are shown in figures 3 and 4. It is possible to observe that there was agreement between both instruments in the task of SRT as in the CRT, which demonstrates the validity of RTST in all groups evaluated.

Figure 3 Validity of RTST compared to the reference test for SRT. 

Figure 4 Validity of RTST compared to the reference test for CRT. 

The results concerning the reliability of the instrument by means of test-retest procedure are shown in Table 2.

Table 2 Analysis of RTST (ANOVA and ICC) reliability and descriptive values ​​relating to SRT and CRT (milliseconds) in the test and retest. 

Groups Test Minimum/ Maximum Mean ± DP Errors ANOVA CCI IC95%
G1 STR1 233 - 368 308.03 ± 33.627 2 F=.579 p=.579 CCI=.810 p=.000 .60 - .91
STR2 207 - 359 304.23 ± 34.363 1
CRT1 404 - 554 474.97 ± 41.001 6 F=1.583 p=.218 CCI=.868 p=.000 .72 - .94
CRT2 398 - 576 468.50 ± 42.038 3
G2 SRT1 204 - 405 304.27 ± 46.334 3 F=.211 p=.649 CCI=.821 p=.000 .63 - .91
SRT2 228 - 398 301.40 ± 40.184 1
CRT1 426 - 599 507.00 ± 45.625 4 F=3.080 p=.090 CCI=.865 p=.000 .72 - .93
CRT2 412 - 555 497.70 ± 40.950 3
G3 SRT1 265 - 447 315.73 ± 43.831 4 F=.531 p= .472 CCI=.808 p= .000 .60 - .91
SRT2 247 - 418 310.80 ± 47.914 4
CRT1 428 - 629 526.03 ± 54.546 4 F=2.142 p= .154 CCI=.802 p= .000 .59 - .90
CRT2 411 - 629 514.67 ± 51.655 7

Note 1: test and 2: retest;Errors: mean in anticipation (SRT) and choice (CRT); ± SD: standard deviation; ANOVA: Repeated measures analysis of variance; ICC: intraclass correlation coefficient (p < 0.05); CI95%: confidence interval of 95%.

As shown in table 2, the results for the reliability of RTST in the three groups were significant (p < 0.05) within the reference values ​​considered adequate (ICC > 0.70). ANOVA for repeated measures found no significant differences between test and retest ratings (p > 0.05), confirming the reproducibility of the instrument. It is also possible to observe that the values concerning the number of errors in both tests, and the average error by anticipation (SRT) were lower than the average errors of response choice (CRT).

Discussion

The RTST was designed based on the use of electronic technologies for shooting tasks simulation. The proposed tests allow the evaluation of reaction times in simple situations where a single answer is possible, and in more complex situations, when it is necessary to choose the most appropriate answer for making the right decision according to the presented situation.

The instruments developed for the assessment of RT usually consist of software using the computer keyboard or mouse to perform manual tasks. These tasks are often used for the evaluation of RT in different contexts and situations. Studies with swimmers 3, volleyball players 17, soccer players 18, military pilots 19, among other contexts, were found. However, the tasks using the computer keyboard or mouse are not always suited to situations experienced by individuals under evaluation, disregarding the specificity of the activities developed in the context of the studied population.

The construction of a measuring instrument requires the observance of certain fundamental criteria that confirm the results safely, among which we emphasize the procedures reliability and validity 20. A test validity begins at the moment someone considers to build it, and subsists throughout the development, implementation, correction, and results interpretation process 21. As to reliability, it is an integral part of validity and refers to the consistency or repeatability of a measurement. Whether a test is not consistent, i.e., if its repetition does not produce the same results, it will not be reliable 22.

In this study the procedures for validity and reliability of the proposed instrument were performed by following the steps according to previous studies for the development of measuring steps instruments 16,20,22. Concurrent validity analyses showed agreement between the reference test and the RTST, and the paired t test confirmed no significant difference between the two measures. Regarding the analysis of reliability, appropriate values were found related to internal consistency of the instrument, verified by the test-retest procedure for the three groups. Although a value of consensual reference has not been found, in general, ICC values ≥ .70 are considered satisfactory 16. These results are in agreement with a previous study involving the creation procedures, development, and reproducibility of an RT test 11. The authors also found satisfactory reproducibility values for the SRT (ICC = .805) and for the CRT (ICC = .838).

Assessing the differences in the reaction time variable between different age groups was not the aim in this study, but rather to create a valid and reliable instrument to be used by people of different age groups that perform specific tasks related to shooting situations. Accordingly, no instruments that fulfill these particular requirements were found. A study developed by the Research Laboratory of the United States Army evaluated the performance of military in their simple and procedure (choice) reaction time. For this purpose, a battery for computerized tests (The Automated Neuropsychological Assessment Metric - ANAM) characterized by the presentation of visual stimuli on the computer screen and sequential manual responses by pressing keys on a keyboard was used. Although it is used for RT evaluation of the active military army personnel, the battery does not correspond to more specific situations of those professionals' activities, which mainly involves the firing of weapons 12. Another study performed with the Rio de Janeiro military's police of the Special Operations Battalion (BOPE) evaluated the RT through the reaction oculomotor (ruler test). In this test, the individual must catch the ruler, which is released by the evaluator, in the shortest possible time, and the distance the ruler falls is measured. In this case, the evaluation may not reveal accurate data, since it disregards the specific daily demands of professional military police 23.

It is important to highlight that, although a simulation does not have all the components of a real variable environmental situation, one should seek to reproduce the main features of its demands 24. Thus, creating this instrument is justified by scientific research purposes seeking to assess individuals in their most specific contexts so that the results are as close to reality as possible.

Conclusion

Based on the results presented in this study it can be inferred that the software created to evaluate the reaction time in the shooting task is valid and reliable for use in research involving a specific instrument with weapon firing activity contexts.

Further, the instrument created can be used for military training as a simple, low cost and efficient tool that involves speed, accuracy and decision-making task shooting.

References

1 Ribeiro RB, Almeida LS. Reaction time and intelligence: strong data for a week explanation. Aval Psicol. 2005; 4(2):95-103. [ Links ]

2 Lima VF, Oliveira FF, Sinésio T, Vieira MM. Efeitos da prática sistemática do judô no tempo de reação de crianças. Pensar Prát. 2011; 14 (1):1-9. [ Links ]

3 Pereira EF, Teixeira CS, Villis JMC, Corazza ST. Tempo de reação e desempenho motor do nado crawl em diferentes estágios de aprendizagem. Fisioter Mov. 2009; 22 (4):585-594. [ Links ]

4 Rossini JC, Galera C. Stimuli Selection and Analysis in the Visual Search Task. Psicol Refl e Crít. 2007; 21 (1):20-27. [ Links ]

5 Côrtes GG, Gama JFR, Dias AG. Mental processing and reaction time in active aged women practicing against-resistance training compared to aged and young sedentary women. Vértices. 2007; 9 (1):51-60. [ Links ]

6 Corazza ST, Streit IA, Katzer JI, Copetti F, Contreira AR. (2013). Reaction time simple and choice of older drivers: a comparison with respect to sex and physical exercise regularly. Rev. Biomotriz. 2013; 7 (1): 15-27. [ Links ]

7 Silva GP, Pereira VR, Deprá VV, Gorla JI. Reaction time and efficiency of the goalball player in interception/ defense of throwing/attack. Motricidade, 2012; 6(4):13-22. [ Links ]

8 Schmidt RA, Wrisberg CA. Aprendizagem e performance motora: uma abordagem da aprendizagem baseada no problema. Porto Alegre: Artmed Ed. 2010. [ Links ]

9 Morales AP, Azevedo MMA, Maciel RN, Barcelos JL, Neto NTA, Silva VF. Efficacy of the mental processing in volleyball players with differential metacognitive levels. Rev Educ Fís/UEM. 2009; 20 (1):43-50. [ Links ]

10 Capovilla AGS, Capovilla FC, Macedo EC. Validação do software cronofonos para análise de tempo de reação, duração e frequência de segmentação locucionais na leitura em voz alta de itens isolados. Cienc Cogn. 1998; 3 (2):253-340. [ Links ]

11 Pereira EF, Dias JA, Corazza ST. Creation, development and analysis of reproductiveness of test to evaluate simple and choice reaction times. FIEP Bulletin, 2007; 77 (01):613-15. [ Links ]

12 Petra EA, Valerie JR. Age Differences in Simple and Procedural Reaction Time Among Healthy Military Personnel. Human Factors Ergon Soc, 2012; 56 (1):1809-1813. [ Links ]

13 Dias AC, Dantas EHM, Moreira SB, Silva VF. A relação entre o nível de condicionamento aeróbico, execução de uma pista de obstáculos e o rendimento em um teste de tiro. Rev Bras Med Esporte. 2005; 11 (6):341-346. [ Links ]

14 Vickers JN, Lewinski W. Performing under pressure: Gaze control, decision making and shooting performance of elite and rookie police officers. Human Mov Sci. 2012; 31 (1):111-117. [ Links ]

15 Bland JM, Altman DG. Statistical method for assessing agreement between two methods of clinical measurement. Lancet. 1986. [ Links ]

16 Barros MVG, Reis RS, Hallal PC, Florindo AA, Júnior JCF. Análise de dados em saúde. Londrina: Midiograf. 2012. [ Links ]

17 Maciel RN, Morales AP, Barcelos JL, Nunes WJ, Azevedo MMA, Silva VF. Relation between reaction time and specific function in volleyball players. F&P J. 2009; 8 (6):395-9. [ Links ]

18 Ruschel C, Haupenthal A, Hubert M, Fontana HB, Pereira SM, Roesler H. Simple reaction time in soccer players from differing categories and field positions. Motricidade, 2001; 7 (4):73-82. [ Links ]

19 Russo M, Sing H, Kendall A, Johnson D, Santiago S, Escolas S, et al. Visual perception, flight performance, and reaction time impairments in military pilots during 26 hours of continuous wake: implications for automated workload control systems as fatigue management tools. In Strategies to Maintain Combat Readiness during Extended Deployments - A Human Systems Approach. Meeting Proceedings RTO-MP-HFM-124 (pp. 1-16). Available from Available from http://ftp.rta.nato.int/public/pubfulltext/rto/mp/rto-mp-hfm-124/mp-hfm-124-27.pdf . (accessed in June, 2015). [ Links ]

20 Martins GA. Sobre confiabilidade e validade. Rev Bras Gestão Negócios, 2006; 8 (20):1-12. [ Links ]

21 Raymundo VP. Construção e validação de instrumentos: um desafio para a psicolinguística. Letras de Hoje, 2009; 44 (3):86-93. [ Links ]

22 Thomas JR, Nelson JK, Silverman SJ. Métodos de pesquisa em atividade física. Porto Alegre: Artmed. 2007. [ Links ]

23 Santos MR, Filho JF. Profile study of dermatoglyphics, somatotypical and physical qualities of Bopes's team (PMERJ) in year 2005. F&P J. 2007; 6 (2):98-104. [ Links ]

24 Zambon AC, Pereira NA, Politano PR, Delgado SS. Validação e fidedignidade de modelos de dinâmica de sistemas. In: Simpósio de Engenharia de Produção. 2006. [ Links ]

Received: November 14, 2014; Accepted: May 06, 2016

*Corresponding author: Ellen dos Santos Soares. Rua Luiz Castanha, 180. Bairro João Goulart, Santa Maria, RS, Brasil. ellensoa@gmail.com

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License