Acessibilidade / Reportar erro

Creation, validation, and reliability of a shooting simulator instrument for reaction time evaluation

Abstract

"Creation, validation, and reliability of a shooting simulator instrument for reaction time evaluation." The aim of this study was to develop, validate, and verify the reliability of a shooting simulator instrument for reaction time evaluation. 90 Santa Maria Air Base military personnel participated on the study. Software was developed for use with an electronic gun where participants performed two shooting task tests: simple reaction time and choice reaction time. The results of concurrent validity were satisfactory, no significant differences were found between the two instruments and good agreement was observed. The reliability results were significant in both tests. The instrument can be used for research purposes, or for the purposes of military training as a simple, low-cost tool involving speed, accuracy, and decision-making shooting tasks.

Keywords
software; reaction time; military

Introduction

Motor reaction time is a measure widely used in research as an indicator of sensorimotor performance by analyzing the time spent by subjects when performing tasks1. Researchers have investigated the reaction time in different populations, such as children 22 Lima VF, Oliveira FF, Sinésio T, Vieira MM. Efeitos da prática sistemática do judô no tempo de reação de crianças. Pensar Prát. 2011; 14 (1):1-9., adults 33 Pereira EF, Teixeira CS, Villis JMC, Corazza ST. Tempo de reação e desempenho motor do nado crawl em diferentes estágios de aprendizagem. Fisioter Mov. 2009; 22 (4):585-594.,44 Rossini JC, Galera C. Stimuli Selection and Analysis in the Visual Search Task. Psicol Refl e Crít. 2007; 21 (1):20-27., the elderly 55 Côrtes GG, Gama JFR, Dias AG. Mental processing and reaction time in active aged women practicing against-resistance training compared to aged and young sedentary women. Vértices. 2007; 9 (1):51-60.,66 Corazza ST, Streit IA, Katzer JI, Copetti F, Contreira AR. (2013). Reaction time simple and choice of older drivers: a comparison with respect to sex and physical exercise regularly. Rev. Biomotriz. 2013; 7 (1): 15-27., and special populations 77 Silva GP, Pereira VR, Deprá VV, Gorla JI. Reaction time and efficiency of the goalball player in interception/ defense of throwing/attack. Motricidade, 2012; 6(4):13-22.. Reaction Time (RT) is the time interval between the presentation of a stimulus and the beginning of an answer 88 Schmidt RA, Wrisberg CA. Aprendizagem e performance motora: uma abordagem da aprendizagem baseada no problema. Porto Alegre: Artmed Ed. 2010.. Accordingly, it can be simple reaction time (SRT), when there is a single stimulus to be answered with only one possible answer, or choice reaction time (CRT), when there are two or more stimuli to be answered with the most adequate response among several possible alternatives 88 Schmidt RA, Wrisberg CA. Aprendizagem e performance motora: uma abordagem da aprendizagem baseada no problema. Porto Alegre: Artmed Ed. 2010..

In broad terms, the TR varies according to the sex, age, and task complexity. Men in general are faster than women 99 Morales AP, Azevedo MMA, Maciel RN, Barcelos JL, Neto NTA, Silva VF. Efficacy of the mental processing in volleyball players with differential metacognitive levels. Rev Educ Fís/UEM. 2009; 20 (1):43-50., with maximum performance in this variable being reached between 20 and 30 years old and later declining 22 Lima VF, Oliveira FF, Sinésio T, Vieira MM. Efeitos da prática sistemática do judô no tempo de reação de crianças. Pensar Prát. 2011; 14 (1):1-9..

When the reaction time is extremely fast, conducting research with this variable requires the use of specific software to register the time between the presentation of stimuli and the early movements 33 Pereira EF, Teixeira CS, Villis JMC, Corazza ST. Tempo de reação e desempenho motor do nado crawl em diferentes estágios de aprendizagem. Fisioter Mov. 2009; 22 (4):585-594.. Over time, different tools have been developed for the RT assessment using different stimuli, especially visual and auditory, in simple and complex tasks, among which manual tasks such as pressing a mouse button or keys of a keyboard on a computer are the most common 1010 Capovilla AGS, Capovilla FC, Macedo EC. Validação do software cronofonos para análise de tempo de reação, duração e frequência de segmentação locucionais na leitura em voz alta de itens isolados. Cienc Cogn. 1998; 3 (2):253-340.

11 Pereira EF, Dias JA, Corazza ST. Creation, development and analysis of reproductiveness of test to evaluate simple and choice reaction times. FIEP Bulletin, 2007; 77 (01):613-15.
-1212 Petra EA, Valerie JR. Age Differences in Simple and Procedural Reaction Time Among Healthy Military Personnel. Human Factors Ergon Soc, 2012; 56 (1):1809-1813..

Manual tasks have been used in different contexts, however, to evaluate the reaction rate in more specific situations, so it is necessary to create tools that enable the reliable execution of the appropriate tasks in the context in which individuals are embedded. Thus, in situations where specific skills are required, such as in occupations involving the use of firearms, it is appropriate that the evaluation is as close as possible to reality. In the case of military professionals, for example, the skill required to achieve shooting efficiency is acquired by means of learning the fundamentals of shooting in activities where directed attention to important events or objects within a scene in real time is required 1313 Dias AC, Dantas EHM, Moreira SB, Silva VF. A relação entre o nível de condicionamento aeróbico, execução de uma pista de obstáculos e o rendimento em um teste de tiro. Rev Bras Med Esporte. 2005; 11 (6):341-346.,1414 Vickers JN, Lewinski W. Performing under pressure: Gaze control, decision making and shooting performance of elite and rookie police officers. Human Mov Sci. 2012; 31 (1):111-117..

In this regard, it is important that the assessment tool simulate, even in a simplified way, the variable component of reality, and reproduce the real system's main features, which will make the assessment more accurate and reliable. Therefore, the objective of this study was to develop, validate, and establish the reliability of a shooting simulator instrument for the evaluation of reaction time.

Method

The study complied with the standards and criteria required by Law 196/96 of the National Health Council (CNS). The research project was submitted to the Ethics Committee on Human Research of the Federal University of Santa Maria (UFSM) and approved under 16764113.5.0000.5346 protocol. All participants were informed about the research purpose and signed an Informed Consent Form (ICF).

Participants and Procedures

The group was intentionally selected, and was composed of male military personnel (n = 90), between 18 and 50 years old, belonging to Infantry Battalions and Air Police from Santa Maria Air Base in (BASM). The factors adopted as inclusion criteria were as follows: being in the pre-determined age group, accomplishing the same shot tasks related to frequency and type of training, and having not reported diagnoses of disease, or physical or mental limitations that prevented testing.

All military participating in the study underwent the same training developed by BASM every six months. Training activities involved free shot, in which the individual is positioned at a certain distance from a central target with a total of ten weapon shots; and the instant defense shot, in which targets are presented in black, representing the enemy and in white, representing the absence of danger, in which the individual must fire shots only at black targets. The military positioning for these activities can be standing, sitting or squatting.

Assessments occurred on the premises of BASM during working hours. Before starting data collection all participants were briefed and informed about the objectives and procedures adopted in the research, and they completed an interview with identification information. Then an interview about possible variables that could influence the testing, such as sleepiness or quality of sleep at the night before, tiredness, alcohol or caffeine intake, and physical activity practice was performed (Figure 1). After the tests TRS and TRE were performed. These procedures were performed in two assessment days (test and retest), within an interval of 24 hours.

The subjects were divided into three groups (n=30 in each) according to age: G1 (between 18 and 29 years old); G2 (between 30 and 39 years old) and G3 (between 40 and 50 years old).

Figure 1
Sample characterization

Instrument construction

The reaction time in shooting task test (RTST) consisted of specific software using an RF Top Gun electronic wireless pistol compatible with Windows Vista (32bit), XP (32bit), and all types of screen monitors (CRT/LCD/Plasma/Projectors). The software, built on the Java SE 7 platform, regardless of a computer platform, is composed of two tasks (SRT and CRT) using images that are projected on a computer screen as stimuli in the individual evaluation. The main tasks are to shoot (pulling the trigger) as quickly as possible and select the most appropriate response according to the presented stimulus. For the elaboration of the images used in the software, actors were hired and image rights assigned by contract. In to the test, the stimulus is represented by the image of a subject characterized as a criminal, dressed in black, or the image of a victim, dressed in white (Figure 2). The stimuli are presented twenty times for each task, randomly, with a minimum time of 100 milliseconds.

In order to accomplish the tests, a closed room is needed so that the lighting conditions will not interfere with the action of the gun. The test begins with the individual in a shooting position (both arms forward and gripping the gun with both hands), with the gun pointing toward the target located on the computer screen (Figure 2). Minimum body movement is required, since the reaction time does not involve performing movements, but only the onset of response.

Figure 2
Positioning for the test realization and representation of images used as stimuli in the tasks of SRT and CRT.

At the end of each task the software generates the mean time, in milliseconds, from the median 10 attempts (excluding the 5 best and worst times) and further, the number of errors in advance or shots off target, as well as response errors. After calculating the mean value, a time of 84 milliseconds is deducted from mean value due to the transmission error between the output timing of the stimulus of the gun and the arrival of the response on the computer screen, considering the wireless condition of the electronic gun.

Both tasks are described below:

Task 1: Simple reaction time

In this task only the gun trigger is used, being pulled with the index finger of the hand of choice. A single stimulus appears on the screen represented by the image of a criminal, so that, when viewing the stimulus, the individual should pull the trigger as fast as possible. The SRT mean and the number of anticipation errors is generated.

Task 2: Choice reaction time

In this task two buttons are used (the trigger and a button below the trigger of the gun). The image of the criminal or the victim was randomly presented on the screen. When viewing the image of the criminal, the subject should pull the trigger, whereas when seeing the image of the victim, the subject must press the button that is below the trigger of the gun. To develop this task the subject is instructed to place the index finger of one hand (of his choice) on the trigger and the other hand on the button below the trigger. The software calculates the CRT and generates the number of response errors and errors in advance or shots off target.

Statistical Analysis

Data were analyzed using SPSS version 14.0 for Windows with significance level of 5%. For a description of the sample, we used descriptive statistics (mean, standard deviation, and percentage frequency). The normality of variables was verified by the Shapiro-Wilk test. To analyze the validity of the instrument a t test was used for paired samples and the Bland-Altman diagrams for the application of the gold standard method 1515 Bland JM, Altman DG. Statistical method for assessing agreement between two methods of clinical measurement. Lancet. 1986.. We used the intraclass correlation coefficient to test the reliability of the instrument, considering ICC > 0.70 as appropriate values 1616 Barros MVG, Reis RS, Hallal PC, Florindo AA, Júnior JCF. Análise de dados em saúde. Londrina: Midiograf. 2012..

Analysis of Concurrent Validity and Reliability

To establish the validity of RTST the gold standard method was used 1515 Bland JM, Altman DG. Statistical method for assessing agreement between two methods of clinical measurement. Lancet. 1986.. The level of concurrent validity was determined by comparing the measurements obtained in the reaction time test that were created and validated in a previous study, and used as a benchmark to measure the proposed new instrument 1111 Pereira EF, Dias JA, Corazza ST. Creation, development and analysis of reproductiveness of test to evaluate simple and choice reaction times. FIEP Bulletin, 2007; 77 (01):613-15.. A paired samples t test was initially performed to compare the mean values between the two instruments, and finally the Bland-Altman diagrams were produced for the analysis of agreement between the two measurements 1616 Barros MVG, Reis RS, Hallal PC, Florindo AA, Júnior JCF. Análise de dados em saúde. Londrina: Midiograf. 2012..

The reliability was investigated by means of temporal measures stability (reproducibility test, and retest). Testing for both tasks SRT and CRT were performed in two days, with an interval of 24 hours between each evaluation. For the test and retest procedures the environmental conditions were controlled, with both assessments done in the morning at the same place, a closed room with a temperature of 28 ° C.

To analyze the reproducibility of the instrument the intraclass correlation coefficient (ICC) was used based on the parameters of the repeated measures ANOVA 1616 Barros MVG, Reis RS, Hallal PC, Florindo AA, Júnior JCF. Análise de dados em saúde. Londrina: Midiograf. 2012., as shown below:

in which MSs = mean square for subjects

MSe = mean square for error

Results

The 90 subjects were divided into three groups for analyzation (n=30 in each group) according to age: G1 (between 18 and 29 years old); G2 (between 30 and 39 years old) and G3 (between 40 and 50 years old). The data distribution analysis using the Shapiro-Wilk test showed normality in all groups (p> .05).

Regarding concurrent validity, as shown in Table 1, for the three analyzed groups, no significant differences (p <.05) between the two tests in the tasks of SRT and CRT were found.

Table 1
Analysis of concurrent validity (t test for paired samples) and descriptive figures for the SRT and CRT (milliseconds) for the reference test and the RTST

The results of the analysis of agreement between subjects for the three groups are shown in figures 3 and 4. It is possible to observe that there was agreement between both instruments in the task of SRT as in the CRT, which demonstrates the validity of RTST in all groups evaluated.

Figure 3
Validity of RTST compared to the reference test for SRT.

Figure 4
Validity of RTST compared to the reference test for CRT.

The results concerning the reliability of the instrument by means of test-retest procedure are shown in Table 2.

Table 2
Analysis of RTST (ANOVA and ICC) reliability and descriptive values ​​relating to SRT and CRT (milliseconds) in the test and retest.

As shown in table 2, the results for the reliability of RTST in the three groups were significant (p < 0.05) within the reference values ​​considered adequate (ICC > 0.70). ANOVA for repeated measures found no significant differences between test and retest ratings (p > 0.05), confirming the reproducibility of the instrument. It is also possible to observe that the values concerning the number of errors in both tests, and the average error by anticipation (SRT) were lower than the average errors of response choice (CRT).

Discussion

The RTST was designed based on the use of electronic technologies for shooting tasks simulation. The proposed tests allow the evaluation of reaction times in simple situations where a single answer is possible, and in more complex situations, when it is necessary to choose the most appropriate answer for making the right decision according to the presented situation.

The instruments developed for the assessment of RT usually consist of software using the computer keyboard or mouse to perform manual tasks. These tasks are often used for the evaluation of RT in different contexts and situations. Studies with swimmers 33 Pereira EF, Teixeira CS, Villis JMC, Corazza ST. Tempo de reação e desempenho motor do nado crawl em diferentes estágios de aprendizagem. Fisioter Mov. 2009; 22 (4):585-594., volleyball players 1717 Maciel RN, Morales AP, Barcelos JL, Nunes WJ, Azevedo MMA, Silva VF. Relation between reaction time and specific function in volleyball players. F&P J. 2009; 8 (6):395-9., soccer players 1818 Ruschel C, Haupenthal A, Hubert M, Fontana HB, Pereira SM, Roesler H. Simple reaction time in soccer players from differing categories and field positions. Motricidade, 2001; 7 (4):73-82., military pilots 1919 Russo M, Sing H, Kendall A, Johnson D, Santiago S, Escolas S, et al. Visual perception, flight performance, and reaction time impairments in military pilots during 26 hours of continuous wake: implications for automated workload control systems as fatigue management tools. In Strategies to Maintain Combat Readiness during Extended Deployments - A Human Systems Approach. Meeting Proceedings RTO-MP-HFM-124 (pp. 1-16). Available from Available from http://ftp.rta.nato.int/public/pubfulltext/rto/mp/rto-mp-hfm-124/mp-hfm-124-27.pdf . (accessed in June, 2015).
http://ftp.rta.nato.int/public/pubfullte...
, among other contexts, were found. However, the tasks using the computer keyboard or mouse are not always suited to situations experienced by individuals under evaluation, disregarding the specificity of the activities developed in the context of the studied population.

The construction of a measuring instrument requires the observance of certain fundamental criteria that confirm the results safely, among which we emphasize the procedures reliability and validity 2020 Martins GA. Sobre confiabilidade e validade. Rev Bras Gestão Negócios, 2006; 8 (20):1-12.. A test validity begins at the moment someone considers to build it, and subsists throughout the development, implementation, correction, and results interpretation process 2121 Raymundo VP. Construção e validação de instrumentos: um desafio para a psicolinguística. Letras de Hoje, 2009; 44 (3):86-93.. As to reliability, it is an integral part of validity and refers to the consistency or repeatability of a measurement. Whether a test is not consistent, i.e., if its repetition does not produce the same results, it will not be reliable 2222 Thomas JR, Nelson JK, Silverman SJ. Métodos de pesquisa em atividade física. Porto Alegre: Artmed. 2007..

In this study the procedures for validity and reliability of the proposed instrument were performed by following the steps according to previous studies for the development of measuring steps instruments 1616 Barros MVG, Reis RS, Hallal PC, Florindo AA, Júnior JCF. Análise de dados em saúde. Londrina: Midiograf. 2012.,2020 Martins GA. Sobre confiabilidade e validade. Rev Bras Gestão Negócios, 2006; 8 (20):1-12.,2222 Thomas JR, Nelson JK, Silverman SJ. Métodos de pesquisa em atividade física. Porto Alegre: Artmed. 2007.. Concurrent validity analyses showed agreement between the reference test and the RTST, and the paired t test confirmed no significant difference between the two measures. Regarding the analysis of reliability, appropriate values were found related to internal consistency of the instrument, verified by the test-retest procedure for the three groups. Although a value of consensual reference has not been found, in general, ICC values ≥ .70 are considered satisfactory 1616 Barros MVG, Reis RS, Hallal PC, Florindo AA, Júnior JCF. Análise de dados em saúde. Londrina: Midiograf. 2012.. These results are in agreement with a previous study involving the creation procedures, development, and reproducibility of an RT test 1111 Pereira EF, Dias JA, Corazza ST. Creation, development and analysis of reproductiveness of test to evaluate simple and choice reaction times. FIEP Bulletin, 2007; 77 (01):613-15.. The authors also found satisfactory reproducibility values for the SRT (ICC = .805) and for the CRT (ICC = .838).

Assessing the differences in the reaction time variable between different age groups was not the aim in this study, but rather to create a valid and reliable instrument to be used by people of different age groups that perform specific tasks related to shooting situations. Accordingly, no instruments that fulfill these particular requirements were found. A study developed by the Research Laboratory of the United States Army evaluated the performance of military in their simple and procedure (choice) reaction time. For this purpose, a battery for computerized tests (The Automated Neuropsychological Assessment Metric - ANAM) characterized by the presentation of visual stimuli on the computer screen and sequential manual responses by pressing keys on a keyboard was used. Although it is used for RT evaluation of the active military army personnel, the battery does not correspond to more specific situations of those professionals' activities, which mainly involves the firing of weapons 1212 Petra EA, Valerie JR. Age Differences in Simple and Procedural Reaction Time Among Healthy Military Personnel. Human Factors Ergon Soc, 2012; 56 (1):1809-1813.. Another study performed with the Rio de Janeiro military's police of the Special Operations Battalion (BOPE) evaluated the RT through the reaction oculomotor (ruler test). In this test, the individual must catch the ruler, which is released by the evaluator, in the shortest possible time, and the distance the ruler falls is measured. In this case, the evaluation may not reveal accurate data, since it disregards the specific daily demands of professional military police 2323 Santos MR, Filho JF. Profile study of dermatoglyphics, somatotypical and physical qualities of Bopes's team (PMERJ) in year 2005. F&P J. 2007; 6 (2):98-104..

It is important to highlight that, although a simulation does not have all the components of a real variable environmental situation, one should seek to reproduce the main features of its demands 2424 Zambon AC, Pereira NA, Politano PR, Delgado SS. Validação e fidedignidade de modelos de dinâmica de sistemas. In: Simpósio de Engenharia de Produção. 2006.. Thus, creating this instrument is justified by scientific research purposes seeking to assess individuals in their most specific contexts so that the results are as close to reality as possible.

Conclusion

Based on the results presented in this study it can be inferred that the software created to evaluate the reaction time in the shooting task is valid and reliable for use in research involving a specific instrument with weapon firing activity contexts.

Further, the instrument created can be used for military training as a simple, low cost and efficient tool that involves speed, accuracy and decision-making task shooting.

References

  • 1
    Ribeiro RB, Almeida LS. Reaction time and intelligence: strong data for a week explanation. Aval Psicol. 2005; 4(2):95-103.
  • 2 Lima VF, Oliveira FF, Sinésio T, Vieira MM. Efeitos da prática sistemática do judô no tempo de reação de crianças. Pensar Prát. 2011; 14 (1):1-9.
  • 3 Pereira EF, Teixeira CS, Villis JMC, Corazza ST. Tempo de reação e desempenho motor do nado crawl em diferentes estágios de aprendizagem. Fisioter Mov. 2009; 22 (4):585-594.
  • 4 Rossini JC, Galera C. Stimuli Selection and Analysis in the Visual Search Task. Psicol Refl e Crít. 2007; 21 (1):20-27.
  • 5 Côrtes GG, Gama JFR, Dias AG. Mental processing and reaction time in active aged women practicing against-resistance training compared to aged and young sedentary women. Vértices. 2007; 9 (1):51-60.
  • 6 Corazza ST, Streit IA, Katzer JI, Copetti F, Contreira AR. (2013). Reaction time simple and choice of older drivers: a comparison with respect to sex and physical exercise regularly. Rev. Biomotriz. 2013; 7 (1): 15-27.
  • 7 Silva GP, Pereira VR, Deprá VV, Gorla JI. Reaction time and efficiency of the goalball player in interception/ defense of throwing/attack. Motricidade, 2012; 6(4):13-22.
  • 8 Schmidt RA, Wrisberg CA. Aprendizagem e performance motora: uma abordagem da aprendizagem baseada no problema. Porto Alegre: Artmed Ed. 2010.
  • 9 Morales AP, Azevedo MMA, Maciel RN, Barcelos JL, Neto NTA, Silva VF. Efficacy of the mental processing in volleyball players with differential metacognitive levels. Rev Educ Fís/UEM. 2009; 20 (1):43-50.
  • 10 Capovilla AGS, Capovilla FC, Macedo EC. Validação do software cronofonos para análise de tempo de reação, duração e frequência de segmentação locucionais na leitura em voz alta de itens isolados. Cienc Cogn. 1998; 3 (2):253-340.
  • 11 Pereira EF, Dias JA, Corazza ST. Creation, development and analysis of reproductiveness of test to evaluate simple and choice reaction times. FIEP Bulletin, 2007; 77 (01):613-15.
  • 12 Petra EA, Valerie JR. Age Differences in Simple and Procedural Reaction Time Among Healthy Military Personnel. Human Factors Ergon Soc, 2012; 56 (1):1809-1813.
  • 13 Dias AC, Dantas EHM, Moreira SB, Silva VF. A relação entre o nível de condicionamento aeróbico, execução de uma pista de obstáculos e o rendimento em um teste de tiro. Rev Bras Med Esporte. 2005; 11 (6):341-346.
  • 14 Vickers JN, Lewinski W. Performing under pressure: Gaze control, decision making and shooting performance of elite and rookie police officers. Human Mov Sci. 2012; 31 (1):111-117.
  • 15 Bland JM, Altman DG. Statistical method for assessing agreement between two methods of clinical measurement. Lancet. 1986.
  • 16 Barros MVG, Reis RS, Hallal PC, Florindo AA, Júnior JCF. Análise de dados em saúde. Londrina: Midiograf. 2012.
  • 17 Maciel RN, Morales AP, Barcelos JL, Nunes WJ, Azevedo MMA, Silva VF. Relation between reaction time and specific function in volleyball players. F&P J. 2009; 8 (6):395-9.
  • 18 Ruschel C, Haupenthal A, Hubert M, Fontana HB, Pereira SM, Roesler H. Simple reaction time in soccer players from differing categories and field positions. Motricidade, 2001; 7 (4):73-82.
  • 19 Russo M, Sing H, Kendall A, Johnson D, Santiago S, Escolas S, et al. Visual perception, flight performance, and reaction time impairments in military pilots during 26 hours of continuous wake: implications for automated workload control systems as fatigue management tools. In Strategies to Maintain Combat Readiness during Extended Deployments - A Human Systems Approach. Meeting Proceedings RTO-MP-HFM-124 (pp. 1-16). Available from Available from http://ftp.rta.nato.int/public/pubfulltext/rto/mp/rto-mp-hfm-124/mp-hfm-124-27.pdf (accessed in June, 2015).
    » http://ftp.rta.nato.int/public/pubfulltext/rto/mp/rto-mp-hfm-124/mp-hfm-124-27.pdf
  • 20 Martins GA. Sobre confiabilidade e validade. Rev Bras Gestão Negócios, 2006; 8 (20):1-12.
  • 21 Raymundo VP. Construção e validação de instrumentos: um desafio para a psicolinguística. Letras de Hoje, 2009; 44 (3):86-93.
  • 22 Thomas JR, Nelson JK, Silverman SJ. Métodos de pesquisa em atividade física. Porto Alegre: Artmed. 2007.
  • 23 Santos MR, Filho JF. Profile study of dermatoglyphics, somatotypical and physical qualities of Bopes's team (PMERJ) in year 2005. F&P J. 2007; 6 (2):98-104.
  • 24 Zambon AC, Pereira NA, Politano PR, Delgado SS. Validação e fidedignidade de modelos de dinâmica de sistemas. In: Simpósio de Engenharia de Produção. 2006.

Publication Dates

  • Publication in this collection
    Dec 2016

History

  • Received
    14 Nov 2014
  • Accepted
    06 May 2016
Universidade Estadual Paulista Universidade Estadual Paulista, Av. 24-A, 1515, 13506-900 Rio Claro, SP/Brasil, Tel.: (55 19) 3526-4330 - Rio Claro - SP - Brazil
E-mail: motriz.rc@unesp.br