Binding Spatial Location and Stimulus Identity in Short-term Memory

In the working memory model, phonological and visuospatial information are stored by separate and independent systems. However, sometimes binding of visual and verbal information must occur. This study investigated whether these memory systems cooperate in the recall of spatial location of a stimulus defined by both types of information. Participants memorized the spatial locations in which name-and-face pairs were presented and either recalled the position of a test stimulus in which the two types of information were always present (Experiment 1) or recalled the position of a test stimulus that non-predictively contained either the visual or verbal information (Experiment 2). The results showed no cooperation between visuospatial and phonological systems when both types of information were present in the test stimulus. Rather, a clear preference for verbal information was found (Experiment 1). When the test stimulus contained only one type of information, recall based on both verbal and visual information was impaired (Experiment 2). These results suggest that visual and verbal information are not automatically integrated into memory and that storage capacity is smaller for integrated information than for isolated information.


Introduction
studied working memory with special attention on the fractioning of this memory system into basic subsystems specialized for storing different types of information.They first proposed two storage systems and one attentional component.They suggested that the phonological loop was responsible for storing auditory and verbal information, whereas the visuospatial sketchpad was responsible for storing and manipulating the information concerning objects and their spatial relations.The central executive was responsible for coordinating the systems and performing complex tasks (Baddeley, 1986).Experimental evidence clearly indicates a separation between phonological and visuospatial memory (Logie, 1986;Logie, Zucco, & Baddeley, 1990).Variables affecting verbal memory, such as the repetition of meaningless words, do not impair visual memory.Similarly, variables affecting the performance of visual tasks do not affect the performance of tasks dependent on phonological memory.For example, performance in the visuospatial version of Brooks' task (1967) is severely impaired by simultaneously executing a visuospatial tracking task; however, the tracking task does not affect the performance of the verbal version of that same task (Baddeley, Grant, Wight, & Thompson, 1975).This separation is also supported anatomically and functionally.Smith, Jonides and Koeppe (1996) used positron emission tomography to show that verbal tasks mainly activate the left hemisphere, especially Broca's area and the premotor area.Spatial tasks, in contrast, affect the right hemisphere, mainly the ventrolateral frontal, occipital, and parietal cortices.
This working memory model consisting of two storage systems (i.e., the phonological loop and visuospatial sketchpad) has a weakness when it is necessary to integrate the information present in the storage systems and in long-term memory into a single complex representation.In the scope of this model, it is difficult to explain the difference in the memory capacity for words organized into sentences, which can reach 15 words, compared to the memory for unrelated words, which can reach approximately seven words (Baddeley, Vallar, & Wilson, 1987).The model also has difficulty in explaining the capacity for prose, which can reach approximately 20 ideas in a task of immediate recollection (Wilson & Baddeley, 1988).Furthermore, the model cannot explain how phonological memory is affected by the visual similarity between the memorized stimuli (Logie, Della Sala, Wynn, & Baddeley, 2000).The generation of mental images based on the integration of the information present in short-and longterm visual memory, as well as the conscious access to those images (Baddeley & Andrade, 2000), presents another situation in which the working memory model proposed by Baddeley & Hitch (1974) may be lacking.These difficulties, among others, suggest the need for a third storage system -the episodic buffer -which would work to integrate the information present in the different storage systems and in long-term memory into a single complex representation, accessible through conscious awareness (Baddeley, 2000).
The process of information binding, equivalent to the chunking process proposed by Miller (1956), should permit more efficient storage, in which the only multimodal representation contains or refers to the representations present in different modalities.For example, the capacity to store objects defined by the binding of four features (i.e., size, color, orientation, and the presence of a gap) is the same when these features are considered alone, suggesting that visuospatial memory can store up to 16 features when distributed across four objects (Luck & Vogel, 1997).Additionally, the capacity to store objects defined by the binding of different features does not appear to require greater attention than the capacity to store features taken alone, suggesting that binding of color and shape is automatic (Allen, Baddeley, & Hitch, 2006), even when the integration should be performed based on verbal information (Allen, Hitch, & Baddeley, 2009).
In addition to being automatic, binding may occur incidentally in some situations (i.e., it occurs even if irrelevant for the ongoing task).For example, Jiang, Olson and Chun (2000), Olson and Marchuetz (2005), and Olson et al. (2004) showed that the act of recognizing a visual stimulus is hindered if the position of the test stimulus is different from the position in which the stimulus had been memorized.This suggests that spatial information is coded incidentally with visual information, despite participants having been instructed to ignore the position in which the visual stimuli were presented.Our laboratory results showed that the recall of the spatial position of visual objects in a sequence is affected if new objects are presented as replacements of those presented in the memorized positions, thus suggesting that visual and spatial information are automatically coded, regardless of being irrelevant to the ongoing task (Corder & Galera, 2009).Maybery et al. (2009) also showed that performance is affected by the spatial position in which the auditory stimulus is presented in a recognition task for auditory information.Auditory test stimuli presented in the same position in which they had been memorized are recognized better than test stimuli presented in positions different from the ones memorized, thus suggesting that auditory and spatial information were coded in an integrated way, regardless of the task requirements (Experiments 3 and 4 in Maybery et al., 2009).This evidence suggests that memory for visual and verbal events also contains information about the spatial location in which those events occur, just as the memory for a location contains information about the events that occurred at that location.
In the present study, we investigated whether memory for binding between location and the identity of an event is facilitated if the identity of the event is defined by both verbal and visual information.Recent studies have shown that the information about spatial location significantly contributes to the recollection of numerical sequences (Darling & Havelka, 2010).The supposition that the availability of visual and verbal information could be beneficial for memory is also supported by the dual code theory proposed by Paivio (1971Paivio ( , 1986)).Thompson and Paivio (1994), for example, showed that recall in a short-term memory task, with stimuli defined in visual and auditory terms, is better than the recall of stimuli defined only in one of these modalities.This occurs even in a situation in which the participant is unaware that a memory task is being performed (i.e., incidental learning).According to the dual code theory, an event is coded in the memory as a cluster of modalities that could be recovered separately, independent of each other.Therefore, events defined in two modalities would be more likely to be remembered than the stimuli defined in a single modality because bimodal stimuli would have been double-coded.
We attempted to determine whether the availability of verbal and visual information would be able to improve performance on a spatial location task (Walker, Hitch, & Duroe, 1993), in which the participant is instructed to memorize four stimuli presented in random temporal and spatial order on a monitor screen and, after a retention interval, recall the original location of the presented test stimulus.Performing this task requires the bound storage of the location and identity of four events until the test stimulus is presented.Therefore, we sought to determine whether the availability of verbal and visual information associated with a spatial location could improve the recall of that location compared with when only one sensory modality was used.
The spatial location task used in the present study is quite appropriate to the studied theme because it permits the evaluation of the binding of the location with visual and verbal information through the same task, as well as their temporal features.Results from previous studies with the spatial location task are usually marked by a recency effect and by the lack of primacy.The recency effect obtained in these studies changes with the stimuli.Visual stimuli produce a recency effect limited to the last stimulus in the memorized series, whereas verbal stimuli produce a recency effect extended to the penultimate item (Galera & Fuhs, 2003;Walker et al., 1993).In the present study, schematic drawings of faces were used as visual stimuli.Proper names, object names, and pseudowords were used as verbal stimuli.Familiar names and pseudowords were used with the purpose of establishing two levels of difficulty for verbal memory.The memory capacity for pseudowords is smaller than for common words (Gathercole, Pickering, Hall, & Peaker, 2001), and this could make it easier to show a possible cooperation between visuospatial and phonological systems.

Experiment 1
In this first experiment, the goal was to determine any possible cooperation between the visuospatial and phonological systems by comparing performance in the spatial location task using visual stimuli, verbal stimuli, and both visual and verbal stimuli in different blocks of trials.If the visual and verbal information could be incidentally integrated, then we expected better performance on the spatial location task when both types of information were presented for memorization.

Participants
The participants were 24 students (graduate and undergraduate) of both genders (13 women; 11 men), with ages ranging between 18 and 35 years.All participants spoke Portuguese and had normal or corrected-tonormal vision and hearing.Participants were randomly divided into three groups, and each group received one type of phonological information (proper names, object names, and pseudowords).After providing written consent, the participants from each group completed an experimental session consisting of three blocks of trials, lasting approximately 50 minutes.

Materials
Visual and verbal stimuli were used, both individually and combined (visual-verbal).Visual stimuli consisted of face schemes with different formats of the face, eyes, and nose.The combination of four types of eyes, four types of faces, and four types of noses permitted the creation of 64 visual stimuli.Verbal stimuli included proper names, object names, and pseudowords taken randomly from a list of 64 stimuli, which was provided using earphones.The visual-verbal stimuli consisted of a random combination of a face with a verbal stimulus.The visual stimuli were presented in black on a white screen on a 17-inch computer monitor with 800 × 600 pixel resolution.The locations in which the stimuli were presented were marked by four square frames, with sides measuring 4.0 cm with 1 mm black outlines.The frames were horizontally aligned in relation with the vertical center of the screen.The frames were set sideby-side with a 2 cm space between each one.The test stimulus was presented in an additional frame, located in the center of the screen; 6 cm below the positions, the actual stimuli were presented (Figure 1).The stimuli were observed at a distance of approximately 60 cm.The auditory stimuli had a duration of approximately 250 ms and were accompanied by a frame with a 3 mm outline presented in one of the positions mentioned above.E-Prime 1.2 (Schneider, Eschman, & Zuccoloto, 2002) was used to control the presentation of the stimuli and to record the responses.

Procedure
Each trial began with the presentation of the five frames on the monitor screen, four for the stimuli to be memorized and one for the test stimulus (Figure 1).The frames remained on the screen during the entire trial.The stimuli presentation began after 500 ms.Each stimulus for memorization was shown for 1 s, with a 500 ms interval between each stimulus.When only auditory stimuli were presented, each stimulus was accompanied by the frame with a thicker outline presented in one of the positions; that frame remained in the position for 1 s.When only visual stimuli were presented, the faces were presented without sound in different positions.When auditory and visual stimuli were combined, the associated visual stimulus remained on the screen for 1 s.The stimuli were presented in a semi-random spatio-temporal order, with the restriction that the test stimulus appeared with the same frequency in each serial position and evenly in each of the four spatial locations.The test stimulus was presented 500 ms after the last stimulus in the sequence; it was shown in a neutral position and remained on the screen until the participant stated in which location that stimulus had been presented previously.The participants gave their response about the test stimulus location by pressing the letter A (test positioned on the left side of the screen), S (test positioned on the center-left side), D (test positioned on the center-right side), or F (test positioned on the right side) on the computer keyboard.Before beginning the tasks, the participants received verbal instructions and provided written consent.After receiving the instructions, the participants underwent training with 20 trials.
Three experimental factors were manipulated.The type of phonological information (i.e., proper names, object names, and pseudowords) was manipulated among groups.In each group, the type of stimulus in the spatial location task (i.e., visual, verbal, or visual-verbal) was manipulated among blocks, each containing 100 trials; the order by which the participants underwent the different blocks of trials was randomized.The serial position of the test stimulus in the memorized sequence, with four levels, changed randomly from trial to trial.The dependent variable was the percentage of correct responses.

Results
Analysis of variance (ANOVA) performed for an initial analysis of the results considered the phonological information (proper names, object names, and pseudowords), manipulated among groups, with the type of stimulus in the spatial location task (visual, verbal, and visual-verbal) and serial position of the teststimulus in the presentation sequence (1 st , 2 nd , 3 rd and 4 th ) as the repeated measures.This analysis showed that performance was not different among groups that worked with proper names (81%) and object names (80%) and that both had better performance than the group that worked with pseudowords (72%) (F 2,21 = 3.97, p < .035,η 2 p = .27).Based on this analysis, the data regarding the participants who worked with proper names and object names were collapsed.The following analysis considered two levels for the phonological information factor (names and pseudowords), manipulated between groups with 16 and 8 participants, respectively.
Figure 2 shows the serial position curves obtained from the results of these two groups for the three types of stimuli used in the spatial location task.Performance was better in the verbal (84%) and visual-verbal (82%) tests compared with the visual tests (65%) (F 2,44 = 29.04,p < .00,η 2 p = .56).No significant difference was found between the performance in the verbal and visualverbal tests (p = .65,Tukey test).This suggests that the availability of both types of information, visual and verbal, did not contribute to improved performance in the spatial location task.
Performance was also significantly affected by the interaction between the phonological information and the type of stimulus (F 2,44 = 5.11, p = .010,η 2 p = .18).The performance of the group that memorized pseudowords was significantly worse in the verbal (76%) and visualverbal (73%) trials than the performance of the group that memorized names, whose performance in these conditions was 88% and 87%, respectively.As expected, the performance of both groups was the same in the trials with visual stimuli (66% of correct responses).
Performance was strongly affected by the serial position of the test stimulus in the memorized sequence (F 3,66 = 80.34, p < .00,η 2 p = .78).This effect was dependent on the phonological information (F 3,66 = 7.74, p < .001,η 2 p = .26).The performance of the participants who memorized names and pseudowords was approximately the same for the last two stimuli in the memorized sequence, but a clear advantage was observed for the group that worked with names in the first two serial positions.This suggests a more efficient recitation process for names than for pseudowords.This advantage of words could also be explained by preexisting long-term memory associations or by the involvement of different cortical areas in the maintenance of words and pseudowords (Fiebach, Riessman, & D'Esposito, 2006).
The serial position curve also depended on the type of stimulus (F 6,132 = 3.35, p = .004,η 2 p = .13). Figure 2 shows that the difference in the percentage of correct responses between the last and penultimate serial position was smaller in the tests with verbal and visual-verbal stimuli (approximately 10% and 13%, respectively).In the visual tests, however, a 26% difference was found in performance in the trials in which the test stimulus was the last and penultimate stimulus to be memorized.Finally, the three-way interaction, involving the three investigated factors, was also significant (F 6,132 = 3.68, p = .001,η 2 p = .14).This interaction basically refers to the differences in the serial position curves of the trials with visual and verbal stimuli.The verbal and visual-verbal trials had a tendency toward a primacy effect, in which the performance of the first serial position was higher and close to that of the third position for participants who worked with names, but not for those who worked with pseudowords.In the tests with visual stimuli, no primacy tendency was found.As expected, performance in the trials with visual stimuli did not depend on the type of verbal information.

Experiment 2
Results from the first experiment clearly showed a preference for using verbal information, when available.In that experiment, we used a design in which the visual and verbal information was consistently associated with the memorized stimuli and both types of information were present in the test stimulus.This allowed the participants to provide their answers exclusively based on verbal information in the verbal and visual-verbal conditions.Experiment 2 investigated the capacity of storing multimodal information in a situation in which the visual and verbal information was associated with spatial locations, but only one of them was present in the test stimulus.Additionally, the information present in the test stimulus changed randomly from trial to trial, which prevented the participant from knowing beforehand any information that would be presented in the test stimulus, thus requiring participants to memorize both types of information (verbal and visual) associated with the spatial locations.Similar to Experiment 1, the phonological information was also defined in three categories: names of people, names of objects, and pseudowords.The difference between Experiment 2 and Experiment 1 was that in all trials both visual and verbal information was presented for memorization associated with spatial location, but the test stimulus had only one type, either visual or verbal.

Participants
The 12 participants of this experiment were undergraduate and graduate students, of both genders, with ages between 18 and 35 years.None of the participants had participated in Experiment 1.

Materials
This experiment used the same material and stimuli (i.e., visual and verbal) as those used in Experiment 1.

Procedure
The spatial location task and the procedure were similar to those of the visual-verbal condition in Experiment 1. participants memorized the spatial locations in which the visual-verbal pairs were presented and, after the retention interval, were confronted with the test stimulus, which contained only one type of information, either visual or verbal.Therefore, the participant, because of the uncertainty of what would be asked in the test stimulus, would code both the visual and verbal information.Similar to Experiment 1, the verbal information could belong to one of three phonological categories: proper names, object names, or pseudowords.Each participant underwent 20 training trials and 100 valid trials.In half of the trials, the test stimulus only had visual information, and the other half contained verbal information.The type of phonological information and the serial position of the test stimulus in the memorized sequence were randomized across trials.

Results
ANOVA was performed on the rate of correct responses for each participant in each experimental condition considering three factors manipulated between participants: type of test stimulus (visual, verbal), phonological information (proper name, object name, and pseudoword), and the serial position of the test stimulus in the memorized sequence.Similar to Experiment 1, the participant's performance was better when the test stimulus was verbal (74%) compared with when it was visual (53%) (F 1,11 = 95.91,p < .00,η 2 p = .89).Performance did not vary as a function of phonological information (F < 1), with a mean correct answer rate of 64% for names (proper and object names) and 62% for pseudowords (Figure 3).
To compare the results of both experiments, the data were regrouped and analyzed considering two levels of phonological information: names and pseudowords.In this analysis, performance was better in the verbal trials (74%) than in the visual trials (53%).Remembering also varied according to the serial position of the test stimulus in the memorized sequence (F 3,33 = 51.99,p < .001,η 2 p = .99).Similar to Experiment 1, the difference in performance between the first and penultimate serial positions was smaller in the verbal trials (18%) than in the visual trials (41%).
When compared with Experiment 1, the results of Experiment 2 showed a generalized performance reduction.ANOVA comparing the performance of participants who worked with names in Experiment 1 with the performance in trials in which names were also used as the test stimulus in Experiment 2 showed a general reduction of 12% in performance in Experiment 2 (F 1,26 = 20.4,p < .001,η 2 p = .44).This decrease in performance was approximately the same for visual (12%) and verbal stimuli (13%) (F < 1).The same comparison was made with pseudowords, which also showed a general reduction in performance of approximately 8% in Experiment 2 in trials that used pseudowords (F 1,18 = 6.86, p = .017,η 2 p = .27).In this case, the bias appeared to be restricted to the visual trials (14%), but the interactions between experiments and type of stimuli did not reach statistical significance (F 1,18 = 2.61, p = .124, η 2 p = .12).

Discussion
In the present study, we investigated whether memory for binding between location and verbal and visual identity of an event would be facilitated if the identity of the event was defined by the two types of information.The literature shows the involvement of separate systems for processing and storing information in these modalities (De Renzi & Nichelli, 1975;Logie, 1986;Logie et al., 1990), but this information must be combined at some moment (Baddeley, 2000(Baddeley, , 2007;;Logie & van der Meulen, 2009).Under some conditions, the information of different systems can be incidentally bound (Corder & Galera, 2009;Jiang et al., 2000;Maybery et al., 2009;Olson et al., 2004;Olson & Marchuetz, 2005).The results of Experiment 1 showed that visual and verbal information did not cooperate to boost the memory for spatial location.When both types of information were available, a clear preference for verbal information was found, with no evidence of incidental binding between verbal and visual information.When verbal information was available, visual information was not considered, not even in situations in which the verbal information had no meaning (pseudowords).Compared with Experiment 1, the results of Experiment 2 showed a generalized reduction in performance when both types of information needed to be coded.Not only did the systems not cooperate, they also appeared to compete for limited processing resources.
In Experiment 1, performance in the location task was affected by the type of verbal information.Performance with names was better than with pseudowords.According to Ward, Avons and Melling (2005), memorizing pseudowords involves purely shortterm memory aspects, whereas the other words would be favored by attachments with long-term memory.This advantage of the name over pseudoword did not occur in Experiment 2. Performance with names showed a reduction of approximately 13% from Experiment 1 to 2, but performance with pseudowords presented an insignificant 3% loss.This interaction between the type of verbal information and the design of the experiments suggests the existence of different strategies in situations in which the participants work in homogeneous situations (Experiment 1) and in situations in which the type of stimulus changes from trial to trial (Experiment 2).Generally, the serial position curve obtained with pseudowords was found to have the same profile as that obtained with familiar names, suggesting that the same recitation process was used for both types of stimuli.
The reduction in performance observed in Experiment 2 may have been caused by the attention given to coding and storing the combined verbal and visuospatial information.The fact that the same loss in performance was observed in both the visual and verbal tests suggests that resource limitation is external to the involved stores and thus has the same effect on both.Some studies, based on the effect of secondary tasks, suggest that visuospatial memory and verbal memory do not share attentional resources (Cocchini, Logie, Della Sala, MacPherson, & Baddeley, 2002;Luck & Vogel, 1997;Scholl & Xu, 2001).The processing capacity limitation does not appear to be caused by aspects specific to each of these systems, but rather by a general limitation in short-term memory information processing (Jolicoeur, 1999;Morey & Cowan 2004).
An ongoing study (Godoy & Galera, unpublished results) has shown that recognition memory for visualverbal binding is more affected by a task of backward counting in steps of three than the memory for visual or verbal information alone.This supports the hypothesis that general system resources are involved in storing integrated visual and verbal information.Furthermore, indirect evidence suggests that attention can be involved in the process of integrating features from different systems of the working memory model.Prabhakaram, Narayanan, Zhao and Gabrieli (2000), in a brain imaging study, verified that the binding of verbal and spatial information activated the frontal region, which is also involved in the functioning of the central executive.The authors explained their results in terms of an episodic buffer, originally proposed by Baddeley (2000), as a working memory component capable of integrating the information of the visuospatial and phonological subsystems.
Interpreting the recency effect is rather difficult.In some studies, the recency effect has been associated with the capacity of short-term memory.However, this association should be considered with caution because of reports of brain injuries that affected verbal memory capacity, without harming the recency effect, whereas injuries in other areas caused no harm memory capacity but eliminated the recency effect (Della Sala, Logie, Cubelli, Trivelli, & Marchetti, 1998).Although the present study did not investigate the nature of the recency effect, recency is different for verbal and visual stimuli.This can be observed in the difference between the recall of locations referenced by visual and verbal stimuli when the test stimulus was the last or the penultimate stimulus of the memorized sequence.For spatial locations defined by verbal information, the reduction observed when the test stimulus occupied the last and penultimate serial positions was approximately 15% (11% in Experiment 1 and 18% in Experiment 2) but was two-fold greater, approximately 33% (26% in Experiment 1 and 40% in Experiment 2), when the spatial location was identified by visual stimuli.This extended recency effect obtained with verbal stimuli is consistent with previous studies (Galera & Fuhs, 2003;Walker et al., 1993) and suggests that the binding between verbal and spatial information is not restricted to perception, but lasts about 3 seconds, the time interval between the penultimate stimulus and the test stimulus.
The performance obtained when the test stimulus was the first or second stimulus in the memorized sequence depended on the type of stimulus.The recall of these first locations associated with names was approximately 80% in Experiment 1 and suggests that the binding between names and locations can persist for at least for 4.5 seconds, the interval between the fade out of the first stimulus in the sequence and the test stimulus presentation.However, the recall of the first two locations based on pseudowords remained 60%.This long-lasting binding of locations and names may be explained by the preexisting longterm memory associations or by the involvement of different cortical areas in the maintenance of words and pseudowords (Fiebach et al., 2006).
In contrast to the long-lasting binding of verbal and spatial information, the binding between the visual and spatial information appears to be more transient.After an interval of 3 seconds, the interval between the presentation of the penultimate stimulus in the sequence and the test stimulus, the recall of the location based on visual information was correct in approximately 60% of the trials.This result is consistent with that of Allen et al. (2006), whom also reported a recency effect restricted to the last stimulus of a sequence of four stimuli defined by the binding of shape and color.According to Allen et al., the fragility of the binding of color and shape may be attributable to retroactive interference caused by the last items in the sequence.The results obtained in the present study suggest that this fragility, also found with pseudowords, may be associated with the memory capacity of the bound features.
In summary, the present study suggests that binding spatial location and stimulus identity depends on the nature of the bound information, with longlasting binding obtained with verbal information, more specifically with words.Moreover, our results also showed a clear preference for the use of verbal information, when available, and no evidence of incidental binding of verbal and visual information.

Figure 1 .
Figure 1.Diagram of events in a trial with visual and verbal information.

Figure 2 .
Figure 2. Experiment 1. Percentage of correct responses for the groups who memorized the names and pseudowords as a function of serial position of the test stimulus for the three types of stimuli used in the spatial location task (visual, verbal, and visual-verbal).

Figure 3 .
Figure 3. Experiment 2. Percentage of correct responses in trials with names and pseudowords as a function of serial position of the test stimulus (visual, verbal).