Colour in action : evidence for a redundancy signal effect when driving motor responses by combined colour and spatial cues

Several studies have demonstrated that either chromatic or spatial information can guide motor behaviour, but so far interactions between these two visual features are little understood. Here we addressed this issue by measuring reaction times (RT) for pointing hand movements which were instructed by either cues for spatial (thought to be predominantly processed in the dorsal visual stream), colour (thought to be predominantly processed in the ventral visual stream) or redundant (combination of colour and spatial information) conditions. While faster responses were found for spatial than for colour cues, most importantly, the shortest RTs were measured for the combined cues (redundancy signal effect, RSE). The data are inconsistent with the predictions of the race model which assumes parallel and independent input from the two streams to the motor system. Instead, the data are better explained by the coactivation model, which proposes combined sensory information from the different stimuli and detection process from the sum of the signals. Here, the redundancy signal effect results from the combination of colour and spatial information. The results provide behavioural evidence for an integration of colour and spatial cues when guiding hand movements.


Introduction
The current view of the functional organization of the visual pathways distinguishes between automated vision for action, in the dorsal pathway (from primary visual cortex (V1) to the posterior parietal cortex) and vision for conscious perception, in the ventral pathway (from V1 to the inferotemporal cortex; Goodale & Milner 1992;James, Culham, Humphrey, Milner, & Goodale, 1995, 2003).The ventral stream deals with explicit holistic descriptions of the visual input (e.g.perceived colour or form), even if such information leads to errors in spatial processing (i.e., visual illusion).Conversely, the dorsal, or sensorimotor stream deals with the metrics of the visual input (i.e.spatial location) relevant for specific actions (such as reaching and grasping).Most importantly, it is proposed to operate outside of consciousness, therefore making it fast (Goodale & Milner, 1992;Goodale & Westwood, 2004;Milner & Goodale 1995; see also Bridgeman, 2000;Pisella & Rossetti, 2000).
However, psychophysical experiments on the functional dissociation between the two streams with respect to perception and action yielded inconsistent results.On one hand, studies on visual illusions showed that information in the visual cortex influences the perceptual judgments (ventral task) but not motor reactions (dorsal task, Aglioti, DeSouza, & Goodale, 1995;Haffenden & Goodale, 1998).On the other hand, in a recent study Franz, Gegenfurtner, Bithoff and Fahle (2000) found that there is no difference between the perceptual and the grasp illusions if the tasks are appropriately matched.
Using a different approach, we wanted to study the interactions between presumed ventral and dorsal tasks by testing the cooperation between spatial location and perceived colour of a stimulus when guiding hand movements.It is known that chromatic information can be used for driving motor responses (Brenner & Smeets, 2004;Katzner, Busse, & Treue, 2006;Pisella, Arzi, & Rossetti, 1998;White, Kerzel, & Gegenfurtner, 2006).However, these studies do not distinguish between the detection of a chromatic contrast and the perception of colour.Whereas the former is possible based solely on dorsal signals (Gegenfurtner et al. 1994;Nassi & Callaway 2006), the latter is presumed to be a ventral domain.We distinguish between the two by having the subjects make decisions based on the perceived colour of a stimulus, rather than the mere presence of a chromatic contrast.We compared the reaction times for motor responses triggered by neutral cues, colour cues, spatial location and their combination (redundancy).To distinguish between stimuli which presumably favour ventral or dorsal stream, we used a variation of a Go/No-Go target detection task, as described by Donders (1968; see also Bartolo, Weisbecker, & Coello, 2007;Katzner et al. 2006).In addition, we modelled our data in order to test the validity of a race model versus a coactvation model, which predict independent, or dependent inputs, respectively, of the inferred dorsal and ventral information to the motor system.

Participants
Nine observers (aged between 20 and 29 years, 4 females, 5 males) participated in the experiments.Five of the observers were naïve.All had normal or corrected to normal visual acuity as tested by the Freiburg Visual Acuity Test (Bach, 1996) and normal colour vision according to the Cambridge Colour Test (Cambridge Research System, CRS).

Apparatus and stimuli
Stimuli were presented on a 19-in colour CRT monitor (Yiama Vision Master pro 514), running at a frame rate of 100 Hz and controlled from the host PC via a Visual Stimulus Generator graphics board (VSG 2/5, CRS).The resolution of the monitor was set at 800x600 pixels.The monitor was gamma corrected using the CRS OptiCAL system.The observers viewed the display binocularly at 57cm distance in a dark room.Their chin and forehead were restrained on a chin and head rest.Observers' responses were recorded by a 4-button response box (CT6 by CRS).The data was analysed off line.The psychophysical procedure was coded in custom made C++ software.
Figure 1 shows a schematic representation of the stimuli and the experimental sequence.The stimuli were presented in the centre of the screen (except for the spatial cues) and had the form of a cross, subtending 0.4 deg of visual angle, with a line width of 2 pixels.During the initial fixation period, the cross was black.In the following test interval, the cross changed its colour (colour cue), position (spatial cue) or both (redundant cue).Table 1 summarizes the different cue combinations.In the case of a colour change of the fixation cross, the colours (see Figure 1 for the colour coordinates) were chosen from the cardinal axes in an equiluminant plane in the Derrington-Krauskopf-Lennie (DKL) colour space (Derrington, Krauskopf, & Lennie, 1984).Care was taken that the coordinates would not exceed the gamut of the monitor.
In the case of a position change of the fixation cross the locations were chosen along the horizontal or vertical retinal meridians at 10 deg eccentricity from the centre (left, right, upwards or downwards).The background was neutral (it appeared grey to the observers), with chromatic coordinates (x = 0.310, y = 0.310, L = 30 cd/ m 2 ) and extended the entire monitor screen.Its luminance matched that of the colour stimuli.

Procedure and instructions
Observers adapted to the background for 5 min prior to the experiment.They initiated each trial by pressing a button on the response box.During the following fixation period of 1500 ms, subjects had to fixate a black central cross, superimposed on the grey background.Thereafter, the actual test stimulus appeared for 100 ms.The stimuli were presented in a Go, No-Go procedure (Donders, 1968), in which Go cues instructed a left or right motor response ("react condition") and No-Go cues instructed no reaction to the stimulus ("no-react condition).In the react conditions, observers were asked to respond as quickly as possible but also as correctly as possible to the stimulus onset.Observers were instructed to respond in the react condition by releasing the initial button and to press the corresponding button on the response box (for example, to press the right button on the response box when the fixation cross turned green).The time difference between the stimulus onset and the button release event determined the reaction time and was measured with a precision of ±1 ms.The observers initiated the next trial on their will.In the no-reaction condition the participants were instructed to withhold their response by keeping the initial button pressed and no reaction time was measured.After a no-react target the next trial was generated automatically.
One trial was finished after 20 presentations of each cue.One session consisted of 5 trials.Each observer completed altogether 400 presentations in each cue condition.The trials were separated in 2 sessions, on different days.Before the data were collected, subjects were trained in two or more training sessions until they reached 95% or higher correct responses for each condition.The data from the training sessions were discarded.

Task
The conditional presumed ventral task (perceptual route, De Jong, Liang, & Lauber, 1994;Hommel, 1993;Kornblum, Hasbroucq, & Osman, 1990) was a motor response to a particular colour cue; for example: "press the left button on a response box if you see the fixation cross turns red".An unconditional, presumed dorsal task (action route, De Jong et al., 1994;Hommel, 1993;Kornblum et al., 1990) was a response which spatially corresponded to the stimulus location (spatial cues); for example: "press the left response box button if the fixation cross jumps to the left".The interference between colour and spatial cues was measured using a redundant-target paradigm (Miller, 1982(Miller, , 1986;;Mordkoff & Yantis, 1993).Here, responses to the two different sensory signals (colour and spatial cues) were presented simultaneously (redundant cues) and were compared with responses to the single signals.
Prior to the experiment, the observers were instructed about the colour and spatial conditions which were defined as react and no-react targets (see Table 1).In the single colour react condition the fixation cross changed its colour to red or green, instructing left or right motor responses, respectively (see Figure 1B).In the single spatial react condition, the fixation cross displacement to left or right instructed left or right motor responses, respectively (Figure 1A).The condition of special interest for us was the redundant condition, which is shown in Figure 1C.In this case, the position of the fixation cross was changed simultaneously and in congruent manner with the colour change.Left or right motor responses were expected to <<red+left>> or <<green+right>> targets respectively.For instance, a red cross to the left was instructing left motor response.To make sure that eventual RSE was not induced simply from the higher number of sensory signals we implemented the neutral conditions.Here, colour and position of the target changed simultaneously, but one of the sensory inputs was irrelevant (neutral) to the task: for example, the blue cross to the right ("neutral colour" condition) or the red cross upwards ("neutral spatial" condition), as shown in Figure 1D.In this case, the observers were instructed to initiate left or right motor response following the relevant sensory input.The neutral colour appeared equally often in a react and no-react condition.No-react signals were determined by blue or yellow colour changes and displacements of the fixation cross either upwards or downwards.In the noreact instructions there were also single and redundant conditions as shown in Table 1.All conditions were presented in pseudo-randomized order until 20 correct responses had occurred in each condition.

Statistical analysis
Statistical significance for differences between left and right motor responses was evaluated by conducting paired t-tests for each of the participants separately.In the analyses for the redundancy signals effect we examined whether RTs to redundant targets were faster than RTs to single targets.Mean RTs were determined for each cue condition (redundant target, single color, single spatial, neutral colour and neutral spatial).
Here, statistical significance was evaluated by a oneway ANOVA involving the within-subjects-factor of target type.Post-hoc analysis: multiple comparison procedure (MATLAB, Statistics Toolbox) determined which pairs of mean RTs were significantly different.
In the last analysis we compared RT distributions between conditions to test for violations of the race model inequality.The statistical significance was evaluated by conducting t-tests across participants at each of the 10 percentile pairs.

Parallel processing models
Independent parallel processing (Race Model) and coactivation parallel processing (Coactivation Model) The race model for simple RT stipulates that (1) each individual stimulus elicits a detection process performed in parallel to the others and (2) the fastest process (i.e.winner) determines the observed RT (Raab, 1962).This model is based on the assumption that RSE is generated by statistical facilitation: if detection latencies are interpreted as non-negative, random variables, the time to detect the first of several redundant signals is faster, on average, than the detection time for any single signal.
On the other hand, the coactivation model proposes a combination of sensory information from the different stimuli and followed by a detection process based on the sum of the signals (Miller, 1982).Here, RSE results from the combined information: the detection is reached faster for two congruent processes (Figure 2 shows schematic representation of the models).
We modelled our data with the predictions of the race model inequality, which is a standard tool to distinguish between the race and the coactivation models.Testing the race model inequality amounts to testing whether an observed RT speed-up is too large to be attributed to statistical facilitation (via probability summation).In our study we checked the race model inequality for single colour or spatial vs. redundant signals and single + (neutral colour or spatial) vs. redundant signals.We followed Miller (1982Miller ( , 1986Miller ( , 2004)), who proposed a standard testing tool used in many RT studies: P(RT < t|S1 and S2) ≤ P(RT < t|S1) + P(RT < t|S2), (1) Here t is the reaction time needed to respond to a signal and S1 and S2 are the two targets.This inequality stipulates that the RT distribution function for redundant stimuli is never larger than the sum of the RT distributions for the single stimuli.Colonius (1990) has demonstrated that all independent race models obey the race model inequality.An observed violation of inequity (equation 1), therefore implicates some type of coactivation (Miller, 1982(Miller, , 1991)).In the case of a division between action and perception (Milner, 1998;Milner & Goodale, 1995), the redundant targets in our study should be processed with the speed of the faster of the two pathways.On the other hand, any violation of inequity (equation 1), would be at odds with the assumption of strict segregation of action and perception.Instead it would indicate a neural summation (or Coactivation Model) of crossdimensional colour and spatial information.

Inter-subject and intra-subject variability
First we analysed the inter-subject variability of the data by comparing the overall mean RTs between the observers.The statistical evaluation showed a significant difference between the RTs for different observers [F (8 ,1791) = 19; p < .05].For this reason further data analysis was performed for each observer individually rather than using group statistics.The same pattern of variability among the subjects was observed for all other signal conditions.We also compared the within-subject RTs for left and right motor responses in each condition (colour, spatial, redundant or neutral) but found no difference (paired T-test, p > .05).Thus, for further RT comparisons, the left and right motor responses in each cue condition were averaged.

Redundancy signal effect
We compared the RTs for the spatial, colour, redundant and neutral cues.A one-way ANOVA involving the within-subject factor cue revealed a significant effect of that factor [F (4, 395) = 56; p < .05for observer MW and F (4, 395) = 38; p <.05 for observer SK].Post-hoc analysis was performed to determine exactly which pairs of conditions are significantly different.The difference was significant (p < .05)for RTs to spatial vs. RTs to colour (Figure 3, upper left panel) and for RTs to neutral spatial vs. RTs to neutral colour targets (Figure 3, lower left panel).This was true for all subjects.The factor redundant signal was significant (p < .05)for all comparisons.Only for two of the observers did the difference between the RTs for the single spatial condition vs. RTs for the single neutral colour condition reach significance.A comparison across the observers between single colour vs. neutral spatial and single spatial vs. neutral colour conditions is presented in the lower right panel of Figure 3. Thus, across observers, RTs to single colour and spatial neutral signals were significantly longer than those to the single spatial and neutral colour cues (on average 80-100 ms).For all subjects the shortest reaction times were observed for the redundant targets (Figure 3, upper right panel): for instance 317±4 ms for observer MW and 312 ± 7 ms for observer SK.The calculated redundancy signals effect was in the range of 21-32 ms across the subjects.

Race Model Inequality
We tested the race model inequality for single colour or spatial vs. redundant cues and single+(neutral colour or spatial) vs. redundant cues.Figure 4, left plot, shows the calculated cumulative probability density functions (CDF) across the subjects in the single colour, single spatial, redundant condition and the model sum of the two single signal conditions.We compared the CDFs by t-test, at each of the calculated percentiles, and found significant violations of the race model.In particular, statistically significant violations were observed at 0.15, 0.25, 0.35, and 0.45 percentiles (p < .05).Though the violation was not significant at each of the percentiles, the shape of the curves explicitly show that the fastest redundant condition responses can be faster than the fastest single condition.Analogously we compared redundant target CDFs to the sum of the CDFs in the neutral colour and neutral spatial condition (Figure 4, right plot).Statistically significant violations were observed at 0.5, 0.15, 0.25, and 0.35 percentiles (p < .05).

Discussion
We examined the implementation of colour in guiding fast motor responses.We found that colour drives motor responses, whereby the response times to colour targets were longer (on average 410 ms) than those to spatial targets (330 ms).This finding is consistent with a number of other studies which showed that colour is used for driving motor responses (Brenner & Smeets 2004;Schmidt, 2002;White et al., 2006).Our study exceeds these findings by showing a redundancy signal effect which results from the combined processing of colour and spatial information (Figure 4).It is typically found that RTs to redundant cues are faster than RTs to single cues, and this is commonly referred to as redundancy signal effect (Giray & Ulrich, 1993;Iacoboni & Zaidel, 2003;Krummenacher, Muller, & Heller, 2001, 2002;Miller, 1982Miller, , 1986Miller, , 2004)).We observed redundancy signals effect in the range of 21-32 ms across the observers.Testing the race model inequality in single and neutral conditions (see Figure 4) showed that the observed redundancy signals effect did not arise simply from the higher number of sensory signals (the redundant condition) presented to the visual system.Based on this analysis we reject the race model as an explanation of the RSE and we propose instead an explanation based on the coactivation model.But what is the origin of the chromatic signals used for guiding motor actions?Our findings showed that chromatic as well as spatial information is efficient in guiding hand movements and can be integrated, resulting in an even faster response (redundancy effect).This contradicts the results by Cressman, Franks, Enns and Chua and colleagues (2006) that tested efficient intentional pointing in both location and colour change conditions.They found that only targets that changed spatial location elicited automated limb modifications, and therefore only cues from the dorsal pathway seem to contribute.However, their results may be explained by the recent physiological and psychophysical findings which show that neurons in the dorsal pathway (area MT) respond to high chromatic contrast (Gegenfurtner et al., 1994;Schiller & Colby, 1983, see also Nassi et al., 2006;Nassi & Callaway, 2006).Therefore, it is possible for the motor system to take advantage of a colour signal in the dorsal motor pathway.In this context an important difference between these and our experiments has to be stressed: in our experiments, the subjects did not merely detect and respond to any chromatic contrast, but they were required to identify the colour first, before making the decision of an action.The perception of colour is -in contrast to a simple detection task -presumably a task of the perceptually related ventral stream (Goodel & Milner, 1992;James et al., 2003;Livingstone & Hubel, 1988;Milner & Goodale, 1995;Ungerleider & Mishkin, 1982;Zeki 1993).From this we infer that the motor responses in our experiments were driven by colour signals originating in the ventral (perceptual) stream.
One could argue that colour and spatial information in our experiments were both processed and perceived in the ventral stream.In these terms, the measured RTs would presumably reflect the processing times of colour and spatial information in the ventral stream.This would also explain the shorter RTs for the combined cues: it is known that the mixed signals from the magno-, parvo-and koniocellular pathways already exist in early cortical area V1 (Sincich & Horton, 2005).Furthermore, there is growing evidence in the literature for a close interconnection between the two streams.In functional imaging studies about perceived judgements of different visual attributes, such as orientation (Cornette et al., 1999;Faillenot, Sunaert, Van Hecke, & Orban, 2001), direction of motion (Cornette et al., 1998;Ferrera, Rudolph, & Maunsell, 1994) and shape of objects (James et al., 2003), was shown that these attributes are processed in both ventral and dorsal visual pathways.More interestingly, it was also found that when observers had to make active judgements about colour, the dorsal visual stream also cooperated (Claeys et al., 2004).Their experiments showed that with colour as the attribute, successive decision processes which link visual signals to motor response involve both ventral and dorsal visual pathways processing.In yet another study, Katzner et al. (2006) investigated the co-processing of colour and another dorsal feature, i.e. motion.They measured reaction times when participants were required to attend to a particular colour and direction of motion in moving random dot patterns and to report the appearance of the designated targets.They found that reaction times to simultaneous presentations of colour and direction targets were too fast to be reconciled with models proposing separate and independent processing of such stimulus dimensions.They interpret their data as behavioural evidence for an integration of colour and motion signals.Our data are consistent with these studies suggesting a close cooperation between ventral and dorsal visual streams.However the exact location and nature of integration of their signals remains an open question.
In the following paragraphs we will consider a model for the contribution of colour to action.We examined the response times for the combination of colour and spatial signals in order to test whether the two inputs to the motor system are independent.Here, we found significantly faster responses to redundant cues than to either of the cues alone.This redundant signal effect was, across observers, in the range of 21 to 32 ms.We modelled the results by the race model inequality test (see Figure 4) and showed that the race model is inadequate for predicting this findings.Therefore, we propose that the RSE observed in our study is better explained by assuming the coactivation model.This strongly indicates a neural summation which underlies the shortened reaction times in the redundant cue condition.
A coactivation model may also explain a recently demonstrated interaction between spatial and semantic cues for action, in a study by Coello and colleagues (Bartolo et al., 2007).Together, these and our findings imply that different perceptual domains of the ventral stream are integrated through similar mechanisms when processed for action.Thus, the coactivation model may be a possible generic mechanism for interactions between perception and action processes.However, further research will be required to reveal the exact neural circuits underlying the redundancy signal effect as evident by the coactivation model.Better understanding the effect may help elucidate the flow of information for driving motor responses.

Conclusions
In this study we showed that perceptual chromatic signals, presumably from the ventral pathway, can trigger efficiently motor behaviour.We interpret the slower response times in the colour identification task as compared to the spatial location task as evidence for different neural representations encoded in different regions of the visual brain.We found the fastest responses for the redundant cues, suggesting that the combined sensory information reaches the decision criterion for action faster.We showed that the race model failed to adequately explain the RSE.Instead, we explain our data in terms of the coactivation model, which assumes an integration of the different, but congruent visual features.While we cannot distinguish whether colour and spatial cues are processed separately (presumably in the ventral and dorsal pathway) or within the same cortical region (possibly ventral), our findings suggest that they are combined before guiding hand movements.

Figure 1 .
Figure 1.Schematic representation of the experimental sequence and of the stimuli in the different conditions.Panel A shows the spatial condition.Panel B represents the colour condition.Here, the fixation cross changes its colour to red (x = 0.433, y = 0.300, L = 30 cd/m2) or green (x = 0.302, y = 0.548, L = 30 cd/m2).Panel C shows the redundant condition: the colour and the position of the fixation cross were changed simultaneously in a congruent manner.Panel D shows an instance of neutral spatial and neutral colour conditions.Neutral conditions were either yellow (x = 0.348, y = 0.405, L = 30 cd/m2) or blue (x = 0.262, y = 0.201, L = 30 cd/m2) plus a displacement of the fixation cross to the left or to the right.

Figure 2 .
Figure 2. Schematic representation of the different parallel processing models: race and coactivation.

Figure 2 .
Figure 2. RTs in different cue conditions.Data are mean across nine observers, upper left panel: RTs for colour and spatial cues, respectively; lower left panel: RTs for neutral spatial and neutral colour cues, respectively; upper right panel: redundant targets; lower right panel: RTs for single colour and neutral spatial cues and RTs for single spatial and neutral colour cues; Error bars show 1 SEM.

Figure 4 .
Figure 4. Race model.Left plot: single cue conditions.Dots and black solid line: the CDF calculated for the experimental redundant cue condition.Pentagrams and dashed gray line: the sum of CDFs predicted by the race model.Dashed black line and open circles: the CDF calculated for the experimental spatial cue condition.Solid gray line and diamonds: the CDF calculated for the experimental colour cue condition.Right plot: (single + neutral) cue conditions.Dots and black solid line: the CDF calculated for the experimental redundant cue condition.Pentagrams and dashed gray line: the sum of CDFs predicted by the race model.Dashed black line and open circles: the CDF calculated for the experimental "neutral colour" cue condition.Solid gray line and diamonds: the CDF calculated for the experimental "neutral spatial" cue condition.Stars show the percentiles at which the value of the T-test for violation of the race model inequality was highly significant (p <.05 at 0.15, 0.25, 0.35, and 0.45 percentiles).

Table 1 .
Possible combinations of react, no-react and neutral cues.