A robot for verifying the precision of total reaction time measurement

The level of variability in psychomotor behavior and the use of several distinct sets of equipments in Reaction Time (RT) assessments might jeopardize the validity and reliability of such measures. This study presents the development and verification of Emboici Robot—a robot capable of performing accurate RT assessments consisting of response to a visual stimulus by pressing a button—whose purpose is to measure the accuracy of RT assessments. We evaluated the accuracy and precision on four different days, each providing 300 measurements. These assessments generated a RT of 46.95ms (+6.04). No significant effects were found in the RTs obtained and, as a result, there is evidence that the Emboici Robot is stable, reliable, and precise. The robot can be a viable solution for verifying precision and accuracy of any given software with simple RT assessments with visual stimulus requiring as response the pressing of a button or key.


Introduction
Reaction Time (RT) is a simple measure of the time elapsed in milliseconds between the receiving of a stimulus and the onset of a motor response to it (Erickson et al., 2011).This is currently used by modern psychology as a significant method of diagnosis (Zajdel & Nowak, 2007).When we apply a test to measure RT using computer software, we can record the time elapsed between the receiving of a stimulus and the conclusion of a task-this is named Total Reaction Time (TRT).
Since computers are normally associated with precision and certainty-not to mention the possibility of introducing more complex stimuli (Spruyt, Clarysse, Vansteenwegen, Baeyens, & Hermans, 2010)-it is expected that the improvement and availability of platforms used in cognitive computerized testing platforms (such as the RT assessments) allow the application of monitoring tests, producing precise measures and speed of processing (Coppel, 2011).
Many researchers have examined the precision in RT assessments by computer systems (Ohyanagi & Sengoku, 2010).This is a major concern of studies on RT testing, chiefly using computers not specifically designed to manage time with millisecond precision (Myors, 1998) or those requiring additional hardware for the task (Xie, Yang, Yang, & He, 2005).
It is clear that video monitors, the devices used in responding to stimuli, the computers' operational systems, and the software used in presenting the stimuli are technical factors that influence precision (Ohyanagi & Sengoku, 2010).
On account of these setbacks, it may be that data obtained from one computer should not be strictly compared with data from another (Segalowitz & Graves, 1990), or that one test performed in a study with a certain mouse might produce significantly different results in another using a different mouse (Plant et al., 2003).Hence, it poses a problem for establishing valid, accurate measures.
There is commercial software that uses external accessories for recording responses-such as the "Response Panel" in the Vienna Test System (Schuhfried & Prieler, 2005), or the SR-BOX (Serial Response Box) in the E-Prime® (Cavézian et al., 2010)-in order to eliminate the delays already known to be present in the use of mouse devices (Cernich et al., 2007;Chambers & Brown, 2003) or keyboards (Damian, 2010;Neath et al., 2011).
The difficulty of comparing the results obtained from the performance of RT tests in one given system and another (chiefly due to the variability of human beings) calls for a common, consistent element that is accurate and precise, such as a robot, so that these different systems might be matched and analyzed.
Easy-to-use, low-cost sensors with integrated open source boards such as Arduino are being used with reasonable results in studies, e.g. as further encouragement for paraplegic patients to move their limbs (Rush & Acm, 2009), aiding in the daily recovery of patients (Bin Ambar, Bin Mhd Poad, Bin Mohd Ali, Bin Ahmad, & Mahadi bin Abdul Jamil, 2012), or even working in a system that classifies different sorts of lights for the blind (Ando, 2005).
A study by D'ausilio (2012) demonstrated the accuracy and precision of an Arduino microcontroller board-which ensures portability and precision entirely independent of the computer or external software-for typical experiments with psychological and neurophysiological assessments requiring proper control over time and input/output signals.The researcher performs eight different assessments with common input/output evaluation in order to show which operations might lessen precision.
There is also an increasing interest in operations involving the interaction between humans and robots in the areas of medicine, housekeeping, manufacturing, and entertainment (Dongjun, Irene, Yong-Lae, Oussama, & Mark, 2010).
With these inquiries in mind and considering both the variability of psychomotor behavior and the use of different measuring equipment for RT that might compromise the efficacy of such measures, the present study presents the formulation, development, and verification of a solution: the Emboici Robot, a robot built from low-cost components to perform accurate Total Reaction Time assessments consisting of the response to a visual stimulus by pressing a button or key.

Presenting the Emboici Robot
The Emboici Robot (whose name is a reference to the arms of a praying mantis, known in Tupy language as "Emboici") was developed with the intent of measuring the Total Reaction Time (TRT) in millisecond precision (see Figure 1).In searching for a solution to the validation of precision in TRT measurements by software, we reasoned that there must be a solution outside the computer where the software is being used and which could provide less possible variability while still being affordable.
This device also allows data acquisition with no significant deviation, making it possible to monitor the measurement of any given software with visual TRT in different computer hardware specifications.The device thus verifies the amount of precision and stability in measurements made by the software as well as the variation to which the software is prone when used in computers with different hardware.
The monitoring of stimulus alteration is performed by a microprocessor through the use of a light-sensitive photodiode with the same color spectrum as the human eye-370nm (violet) to 750nm (red) (Cassares & Petrella, 2001).The photodiode sends an electric signal representing the color intensity, which in turn is codified into a digital signal by the microprocessor.The microprocessor identifies the expected color and activates a digital servo (with a metal shaft bolted to a propeller) to the point of pressing the button or key, and then turning to its original position and awaiting the next stimulus.
A box was made using Medium-density fiberboard (MDF) to hold the Arduino board and the protoboard (see Figure 1-B), while a second MDF box contained the digital servo (see Figure 1-A), providing support for the metal shaft used in pressing the button or key.These components were assembled in separate boxes for the sake of flexibility.In this way, adjustments can be made to suit different keyboards, buttons, etc.
The diagram illustrating the electronic components inside the Emboici Robot-an Arduino board, a digital servo, and a photodiode-is shown in Figure 2.Besides these components, six resistors were used, as well as a BC548 amplifier, three keys for configuring the robot, and three light-emitting diodes (LED) in the colors red, white, and green.
The robot was assembled with an open source Arduino UNO, a BPW21 photodiode for identifying color alteration in the visual stimulus, and a metal shaft triggered by a digital servo (SC-1267SG), detailed as follows: Arduino UNO.An Arduino board is an open source electronic prototyping platform based on flexibility, with easy-to-use hardware and software (Arduino, 2012;D'Ausilio, 2012), designed to promote physical interaction between computer and environment.For this purpose, it utilizes electronic devices in a simple way, based on free software and hardware (Cavalcante, Tavolaro, & Molisani, 2011).
We connected the Arduino board to a 5 Volt commercial AC-DC power supply.Every sensor was connected to the Arduino board so that it recognized the voltage in the analog signal.The analog signal was then forwarded to the 10 bit analog-to-digital converter (ADC) for data processing (Bin Ambar et al., 2012).
Photodiode (BPW21).We used silicon BPW21 photodiode in order to discern light alteration within the visual stimulus.It was positioned in a hermetically sealed casing with a built-in glass, adequate for color correction and ideal for adjusting the total light exposure (Ando, 2005).
This photodiode (BPW21) is tuned to the sensitivity of the human eye (V λ ) and thus distinguishes between the visible spectrum of 370nm (violet) up to 750nm (red) (Cassares & Petrella, 2001).
Digital servo (SC-1267SG).We used a SC-1267SG Digital Servo manufactured by Savöx.Digital servos are a special kind of direct current motors designed to rotate their axes in a 180-degree arc (Bingol & Aydogan, 2012).The precision achieved with these motors is considered good and is obtained by measuring the width of incoming pulses and the period between these pulses (Dias, Grehs, Mendes, Moura, & Ferrugem, 2011).The digital servo is powered by a 5V supply, provided here by the Arduino board.Arduino programming.The Arduino is programmed in C/ C++ using an IDE (Integrated Development Environment) of its own.This IDE-which is available on Windows and Linuxallows the program to be uploaded to Arduino.The software written for the Arduino is called "sketches" and is saved with the file extension .ino(Arduino, 2012).The ATmega 328 chip assembled in the Arduino handles the program and interacts with the peripherals (Bin Ambar et al., 2012).

Method
The present study presents the formulation, development, and evaluation of a robot (the Emboici Robot) built with lowcost components.The Emboici Robot was designed as a solution to measure accurately Total Reaction Time (TRT) assessments consisting of the response to a visual stimulus by pressing a button or key.
The color alteration in the visual stimulus is monitored by a microprocessor through the use of a photodiode.This photodiode identifies the previously-designated color and triggers the digital servo, which in turn triggers the propeller to rotate the metal shaft up to the point where it presses the response key or button, and then returns to its original position.
For the Emboici Robot to perform a TRT assessment, software or another device was needed to generate the stimulus and measure the response time.In the present study, we opted to use another device to generate a stimulus and to measure the response time with as little variability as possible in terms of hardware and software used.To this end, we assembled a device named "TRT Simulator"-also using an Arduino board-as described below.

TRT simulator (TRTsim) including an Arduino board
We opted to assemble a device whose purpose is to simulate a TRT assessment by lighting a green light-emitting diode (LED), the response to which consists of pressing an on/off button.Since this value stood for the total amount of time elapsed between the stimulus (stimulus-LED) and the pressing of the response key (SR-key), we also measured the time between the Emboici Robot's sensor perceiving the stimulus and the time precisely prior to the triggering of the digital servo.
In order to achieve this, we altered the Emboici Robot's source code so that the red LED is lit before issuing the triggering command to the digital servo.One channel is linked to the cable triggering the light of the stimulus-LED in the TRTsim and another channel is linked to the cable triggering the red LED in the Emboici Robot.Through this configuration we can measure the TRTsim and Emboici Robot response time before the digital servo is triggered (therefore bypassing the mechanical components).The time measured falls below 1ms, or 248.0 μs (see Figure 3-B) Making use of these measures, we were able to overlook the time spent by the Arduino board and the photodiode, considering the time measured by the TRTsim as the same as measured by the Emboici Robot.

Procedure
The first step was to identify the time spent by the TRT Simulator (TRTsim) to record the TRT assessments obtained by the Emboici Robot, through the use of a Tektronix TDS 1002 60MHz 1GS/s oscilloscope.By measuring the time elapsed between the pressing of the SR-key and the fading of the green LED (stimulus-LED), we could determine the TRTsim's precision.One channel from the oscilloscope was connected to the SR-key's power supply and another to the stimulus-LED's power supply, making sure that this measure was under 1ms.The TRT Simulator (TRTsim) consists of an Arduino board with a white LED (to aid in monitoring the functioning and preparation of the TRTsim), a green LED (the stimulus remains on until the on/off key is pressed), an on/off switch (to record the response to the stimulus), and a HanRun HR911105A 10/49 network adapter (to recover the response time recorded by the TRTsim).The Arduino microcontroller was programmed to trigger the green LED at random times, thus generating the visual stimulus the Emboici Robot expects.Stimulus response consists of pressing the on/off key (SR-key).
For studies that require stimulus control with millisecond precision, the use of LEDs might be a satisfying solution, since the effort needed to configure a tiny LED lamp is minimal (Stoet, 2010).In this way, the Arduino microprocessor's delay in μs can be discarded without compromising the recorded measurement by the TRTsim.
A WebService was set up in order to receive TRT data taken by the TRTsim only after the completion of TRT measures, so that the transmission time presents no interference in the TRT recording by the microcontroller.
With this TRT Simulator we have a stimulus generator with inaccuracy of less than 1 millisecond, enabling us to analyze the precision with which the Emboici Robot can operate.

Verifying the execution time of the TRT Simulator
For the verification of time as recorded by the TRT (TRTsim), we used the Tektronix TDS 1002 60MHz 1GS/s oscilloscope, with one channel connected to the cable triggering the green LED (stimulus-LED) and the other connected to the cable receiving a signal from the stimulus response key (SR-key).For each measurement obtained by the TRTsim the value was checked on the oscilloscope (see Figure 3-A).
After we were certain the time spent by the TRTsim would not compromise the TRT assessments made by the Emboici Robot (under 1ms), we began gathering the TRT assessments.
These tests were performed on the 1 st , 5 th , 7 th , and 8 th day (interleaved to ensure uniformity and avoid fatigue) of the same month.On each of these days 300 assessments were made, amounting to 1.200 TRT assessments.Each day the Emboici Robot was used, we redid the robot's setup in the way described below.
There were stops at every 100 TRT assessments made, when the TRTsim would stop the stimulus signaling and wait until the results were transferred.This transfer was carried out through the TRTsim's network adapter.In a computer with network access, we could connect to the WebService address http://10.14.0.111, which in turn was available only when the TRTsim entered on-hold/transfer mode (white LED turned on).The 100 assessments were then stored in TXT files for later evaluation.

TRT assessment
The TRTsim worked this way: the green LED (stimulus-LED) was lit and the on/off switch (SR-key) was pressed.The time elapsed between the stimulus and its response was identi-fied as the total reaction time (TRT).The TRT assessments were stored in the TRT Simulator and uploaded for later evaluation.

Emboici Robot settings
We can see in Figure 4-B the primary control keys for the Emboici Robot: the armKey (1) used in setting up and monitoring the robot arm's position in relation to the SR-key; the colorKey (2) which configures stimulus color; and the startEn-dKey (3) used in initiating and finishing a set of tests.
Each of these keys was designed to fine-tune one of the Emboici Robot's functions.Their settings are described below.

Setting up the arm's position for pressing the on/off switch (armKey)
For the digital servo to press the response key (SR-key) we had to configure the correct position of the Emboici Robot's "arm" (the metal shaft attached to the digital servo).Pressing the armKey made the arm rotate 1 degree clockwise (step = 1º).The armKey was pressed once more when the arm came to the point of pressing the SR-key, so that the final position was stored in the Emboici Robot's memory.The robot arm was connected manually to the bearing gear at approximately 180º (horizontal) in relation to the table surface.Then the armKey was pressed once and the red LED lit up signaling that the arm rotated 1 step clockwise (1º = one degree) at a time until the armKey was pressed again.
Once the SR-key was completely pressed we pressed the armKey to indicate its final position (thus ensuring that the SR-key would be pressed in the TRT assessments).Then, the red LED was turned off and the arm rotated 3º (three degrees) counterclockwise, to what is the initial position for all.

Setting up the stimulus color (colorKey) shown to the photodiode
In order for the photodiode to identify the stimulus alteration, the stimulus color had to be configured (the color chosen in this study was the green).The photodiode connected to the Emboici Robot was positioned above the stimulus-LED.When we pressed the colorKey the color then showing was stored in the Emboici Robot's memory.
In this study we attached the photodiode to the stimulus-LED with electrical tape, enclosing the TRTsim's green LED (see Figure 4-C).
Subsequently, the TRTsim was turned on to light up the stimulus-LED (green LED turned on = stimulus-LED).The colorKey was pressed once (the Emboici Robot's green LED then lit up) and the digital value of the stimulus-LED color was stored (that is to say, the digital value of the stimulus-LED's green color was stored).To finish adjusting the color settings we had to press the colorKey once again (the Emboici Robot's green LED was then turned off).
The Emboici Robot was then started.Every time the stimulus-LED was lit and identified by the photodiode, the robot arm was triggered.

The start/end monitoring key (startEndKey)
For the Emboici Robot to start testing, the startEndKey had to be pressed once.Right after this, the red and green LEDs were lit and the arm pressed the SR-key, then returned to wait for the startEndKey to be pressed again.The TRT Simulator was ready to initiate a set of tests.
When the startEndKey was pressed for the second time, the red and green LEDs went out and the white LED lit up, indicating that the photodiode was waiting for the predefined stimulus.

Data analysis
Descriptive statistics were used to characterize the time performance of robot, and Friedman's test, Intra Class Correlation coefficient and Bland Altman tests to infer the reliability of the robot.

Results
A total of 1200 total reaction time (TRT) assessments were performed by the Emboici Robot over four interspersed days (1 st , 5 th , 7 th , and 8 th days of the same month).On each day 300 TRT performance measurements were taken.The test was generated and controlled by a "TRT Simulator" (TRTsim) built for the sake of measuring the time elapsed between the activation of the stimulus-LED and the pressing of the on/off switch (SR-key).
Descriptive statistics regarding the TRT obtained by the Emboici Robot in each of the four sets of tests are shown in Table 1; such measures were taken in order to evaluate the reliability of the Emboici Robot detailed in this study.At every 100 TRT assessments performed the data were uploaded to a WebService by means of a network adapter.Before and after each set of 100 tests the TRTsim was also rebooted.
There was no significant change in the TRT results obtained in the four days of testing, reporting the time elapsed between identifying the stimulus (green stimulus-LED) and the response (pressing the TRTsim's SR-key) to be an overall average of 46.95ms (SD, 6.04).As we can observe in Table 1, little variation occurred between sets of tests.The average TRT fluctuated 0.29 from the lowest mean (46.71ms 5 th day) to the highest mean (47.00ms 1 st day).The standard deviation was also relatively Note: The measures were stored in a TRT simulator built with an Arduino board and an on/off switch for recording the stimulus response.This stimulus consisted of turning on a green LED prompting the Emboici Robot's photodiode.The robot then reacted by using its arm to press the stimulus response key.
Table 1.Presenting central tendency, range, and standard deviation of TRT (in milliseconds) obtained by the Emboici Robot in four distinct days.low, ranging from 5.84ms to 6.25ms-a difference amounting to 0.41ms.Skewness ranged from 0.115 to 0.164, and the standard deviation for asymmetry (0.141) was the same for all samples.Application of Friedman's test shows that there are not statistically significant changes in the distribution of TRTs measurement over the four days, χ 2 =3.360, df = 3, p = .339.The alpha coefficient for the four items is .855,suggesting that the items have relatively high internal consistency.
The ICC for single measures is much lower at 0.544 than the reliability for all the performance averaged together which is at 0.855 (95% confidence interval = 0.826-0.880)(average measures).
Using the Bland and Altman that discuss the option of using confidence interval bounds, based on the standard error of the mean, for the upper and lower reference lines, and could verify if the two methods are comparable, then differences should be small, with the mean of the differences close to 0 (Table 2).The spread of performance on the day 1 is comparable to the spread on the day 5 (and so far all others combined days).The limits of agreement plotted do fit better, although some skewness remains (Figure 5).
The data collected show that the TRT assessments obtained with the Emboici Robot average the duration of 46.92ms.There is no significant deviation between the samples collected.

Discussion
One thousand and two hundred total reaction time (TRT) assessments were collected by a TRT simulator and performed by a robot named Emboici Robot.This robot was designed to react to a stimulus by pressing a button or key.The measurements were obtained over the period of four different days to ensure that the Emboici Robot was stable.
The present study made use not only of computer software but also of an external accessory (the TRT Simulator) to generate and collect stimulus.We could verify the simulator's accuracy and reliability by monitoring it with a precise oscilloscope.Some studies evaluated the precision in RT assessments made by computer software through the use of external devices; such research, however, typically used the same computer for generating and/or measuring time (De Clercq et al., 2003;McKinney, MacCormac, & Welsh-Bohmer, 1999).
A study by Neath et al. (2011) proposed assembling a device consisting of a photodetector used in identifying changes in brightness on a monitor.This then triggered a solenoid, which in turn pressed the keyboard.The device was used to simulate RT assessments in three different software systems, on two distinct keyboards and two different iMac hardware specifications.As a result, a difference of 5-10ms was found.Using open-source Arduino microcontroller boards Schubert, D'Ausilio, and Canto (2013) demonstrated the reliability, robustness, and precision of this communication in six studies confirmed that the error added to the measurement had an SD of less than 1 ms (Schubert, D'Ausilio, & Canto, 2013).
In the present study, the TRT simulator generates the stimulus, calculates the TRT, and stores the results.This simulator is an autonomous system, composed of its own high-performance microprocessor consisting of no circuit other than its own.Accordingly, the Emboici Robot is another autonomous circuit, powered by a 5V power supply which-as shown in the study of Neath et al. (2011)-was capable of identifying alteration in light stimulus and trigger a digital servo to press a response key.
Analysis of agreement produced by the Bland-Altman's test showed that variability between the combined days is near zero and revealed that measurements obtained by tests meet the expected agreement by each day.
The Emboici Robot can be suitable not only for users with technical expertise, but also those with limited budget, because of its accessible, low-cost components.The Arduino board is a relatively inexpensive option (D'Ausilio, 2012), costing less than a hundred Reais (approximately 45 dollars as of April, 2014).Furthermore, the whole set of components used in assembling the Emboici Robot did not exceed the value of 200 Reais (approximately 90 dollars as of April, 2014).
Researchers in the area of sport psychology and motor behavior show great interest in the performance of reaction time (RT) tests, either among athletes (Hung, Spalding, Santa Maria, & Hatfield, 2004) or in studies examining the influence of sport and physical exercise on athletes and non-athletes (Chan, Wong, Liu, Yu, & Yan, 2011).As Kornspan (2007) notes, one of the first studies to investigate the psychological aspects of sport in a psychology lab in 1894 used RT tests on fencers (Kornspan, 2007  Given the importance of studies making use of RT tests and the variability of RT results obtained by different sets of software and hardware, it is highly recommended that researchers using a computer to collect RT measurements first evaluate the precision and reliability of the chosen platform (Neath et al., 2011).
In short, our study shows evidence that the Emboici Robot is a trustworthy and precise technical resource, capable of properly evaluating tests based on computer software which require responding to a stimulus by quickly and accurately pressing a button or key.

Conclusion
In this study, a robot named Emboici Robot was developed, evaluated, and equipped with a system capable of monitoring color alteration in the visual stimulus generated by a total reaction time (TRT) test.The robot described consists of an Arduino board, a metal shaft attached to a digital servo, and a photodiode capable of monitoring real-time color changes in the visual stimulus.The system we propose is easily assembled and requires only low-cost components.The TRT measurements generated enable the researcher-through the use of the Emboici Robot, whose variability proved to be stable-to evaluate properly any TRT software that makes use of a simple visual TRT test.
Researchers may use this system (Emboici Robot), with these low-cost interfaces, to validate the measurements made by a set of hardware and software in RT tests.Future studies will focus on evaluating some previously consolidated RT software and hardware, aiming to improve software that requires precision in the measurement of time.
Making use of the robot to simulate TRT tests may pinpoint issues in the development of software, as well as identify the most adequate configuration to be used.This serves not only to validate a given software, but also to verify other consolidated software currently in use.

Figure 1 .
Figure 1.Side view of the Emboici Robot.The Emboici Robot was assembled with an open source Arduino board programmed to identify color alteration of a given visual stimulus and to activate a digital servo to press a button or key.Two boxes were made using MDF, one for keeping the digital servo with its propeller (A), and the other for guarding the Arduino board and the protoboard (B).

Figure 2 .
Figure 2. Diagram illustrating the electronic components inside the Emboici Robot.The Emboici Robot was composed of an Arduino board, a digital servo, and a photodiode (besides auxiliary components).

Figure 3 .
Figure 3. Measurements obtained by the oscilloscope during tests made by the Emboici Robot.In (A): a TRT assessment by the Emboici Robotwith channel 1 (CH1) connected to the stimulus-LED and channel 2 (CH2) connected to the stimulus response key (SR-key)-resulting in 47ms.And in (B): a measurement of the time elapsed between the activation of the stimulus-LED and the triggering command sent to the digital servo, amounting to less than 1ms (=248 μs)-after which we can infer that the time recorded with the Emboici Robot is due to the mechanical process through which the digital servo moves to press the SR-key.

Figure 4 .
Figure 4. Presenting the robot and TRT Simulator used in this study.The Emboici Robot in two MDF boxes, A+B, containing an Arduino board, a photodiode sensitive to color alteration, and a digital servo with a metal shaft attached for pressing the on/off switch (SR-key); C+D = TRT Simulator, which consists of an Arduino board, a network adapter, an on/off switch, and an LED for signaling the stimulus.The Emboici Robot contains three control keys: the startEndKey used in initiating and finishing a set of tests; the colorKey for designating the color of the stimulus which will trigger the robot; and the armKey for calibrating the position of the arm.Both sets (A+B and C+D) are supplied by a 5V power supply.

Figure 5 .
Figure 5. TRT sampling generated by the Emboici Robot and Bland-Altman plots of agreement between TRT measurement (milliseconds) in day 1 and day 5 (top left), day 1 and day 7 (top right), day 1 and day 8 (middle left), day 5 and day 7 (middle right), day 5 and day 8 (bottom left), and day 7 and day 8 (bottom right).Central lines indicate the mean differences and external lines indicate the 95% limits of agreement between measures.

Table 2 .
). Test of whether DIFF has a median of 0 with the One-Sample Wilcoxon Test procedure includes the mean and standard deviation of DIFF, Minimum and Maximum values and the significance level for the test that the median of DIFF equals 0.
DIFF= the difference between Day a and Day b .Sig.=significance at level of .05.