Reappearance of Sequences with Different Number of Responses in Extinction and Variation Contexts

ABSTRACT

The reappearance of a response previously extinguished when an alternative response no longer produces reinforcers is called resurgence. The effects of three variables on the reappearance of response sequences - sequence probability (high and low), context of test (extinction and operant variation) and number of responses per sequence (three and five responses) - were investigated. Sequences with high probability reappeared more often than sequences with low probability, the reappearance of the target sequence was more frequent under extinction than under variation, and the reappearance of the target sequence varied inversely with the number of responses by sequence. The reappearance of the target sequence was not conceptualized as resurgence because its frequency was lower than that of the control sequences.

Keywords:
resurgence; sequence probability; context of test; number of responses per sequence; humans

RESUMO

O reaparecimento de uma resposta previamente extinta quando os reforços para uma resposta alternativa são descontinuados é denominado ressurgência. Foram investigados os efeitos de três variáveis sobre o reaparecimento de sequências de respostas: probabilidade da sequência (alta e baixa), contexto de teste (extinção e variação operante) e número de respostas por sequência (três e cinco). Sequências muito prováveis reapareceram mais frequentemente que sequências pouco prováveis, o reaparecimento da sequência alvo foi mais frequente no contexto de extinção do que de variação e o reaparecimento da sequência alvo variou inversamente ao número de respostas por sequência. O reaparecimento da sequência alvo, entretanto, não foi conceitualizado como ressurgência, uma vez que sua frequência foi menor que a das sequências controle.

Palavras-chave:
ressurgência; probabilidade da sequência; contexto de teste; número de respostas por sequência; humanos

Resurgence is the reappearance of a response previously extinguished when an alternative response, recently reinforced, is extinguished. The standard procedure to evaluate resurgence comprises three phases. In the Training Phase, a target response (R1) is reinforced. In the Elimination Phase, R1 is extinguished while another response (R2) is then reinforced. Finally, in the Testing Phase R2, as well as R1, produces no reinforcer. The reappearance of R1, despite the absence of programmed reinforcers, characterizes resurgence (Bruzek, Thompson & Peters, 2009; Epstein, 1985; Sweeney & Shahan, 2016). Variations of this procedure have shown that resurgence occurs not only when R2 is extinguished, but also when there is a reduction in the frequency (Lieving & Lattal, 2003) and delay (Jarmolowicz & Lattal, 2014) of reinforcers. As Jarmolowicz and Lattal emphasize, studies on resurgence have contributed to understand the effects of extinction and history of reinforcers, as well as to prevent the recurrence of problematic behaviors.

In most studies on resurgence the response unit comprises one single response, regardless of whether such response is pressing a lever (e.g., Winterbauer & Bouton, 2010), pecking a disk (e.g., Cançado & Lattal, 2011) or pressing a key on a computer keyboard (e.g., Alessandri, Lattal & Cançado, 2015). Few studies, however, have used behavioral units comprising more responses - for example, units made up by sequences of responses. In one of such studies Bachá-Méndez, Reid & Mendoza-Soylovna (2007, Experiment 2) changed the standard resurgence procedure by exposing rats to four phases in which the reinforcer was always contingent to the emission of a specific sequence of two responses, distributed on two bars (left and right, L and R, respectively). In the first phase, for two subjects the reinforcer was contingent to the emission of the LR sequence, and for the other two it was contingent to the emission of the RL sequence. In the second phase the reinforcer was contingent to the emission of a homogeneous sequence (LL or RR), while the others were under extinction. In the third phase another homogeneous sequence was reinforced so that reinforcers contingent to the emission of LL (or RR) sequence in the previous phase became contingent to the RR (or LL) sequence. The remaining sequences were not followed by reinforcers. In the last phase the reinforcer was contingent to the emission of a heterogeneous sequence opposite to that learned in the first phase. In the third and fourth phases, in which the reappearance of sequences could be observed, the heterogeneous sequence trained in the first phase resurged for all subjects.

Sánchez-Carrasco and Nieto (2005) investigated the resurgence of three-response sequences. In the study rats were distributed in two groups, and different sequences of responses were reinforced in the Training Phase. To one group the RLR sequence was reinforced, while to the other the RLL sequence was reinforced. In the Elimination Phase an alternative sequence was continuously reinforced (LLL) to the two groups, while the sequence of responses learned in the previous phase was under extinction. In the Testing Phase, none of the possible sequences was followed by reinforcers. For all subjects the sequence trained in each group during the first phase (RLR or RLL) reappeared more frequently than the other possible sequences. Reed and Morgan (2006) also observed resurgence of three-response sequences.

In Villas-Bôas’ (2006) study, rats were distributed in two groups and learned to emit four sequences of four responses distributed on two bars. For Group 1 each sequence was trained and then extinguished before the next sequence training; for Group 2 a sequence was extinguished simultaneously to the training of the next sequence (there was no extinction between the training phases). In the last extinction of each group the previously trained sequences reappeared more frequently in Group 2 than in Group 1. However, there was no evidence that the four trained sequences reappeared more frequently than the remaining possible sequences. Therefore, an unquestionable evaluation of the occurrence (or not) of resurgence is not feasible.

Villas-Bôas’ (2006) results are not in line with those found by Bachá-Méndez et al. (2007), Sánchez-Carrasco and Nieto (2005) and Reed and Morgan (2006) who found resurgence of sequences of responses. This inconsistency may be due to procedural factors (e.g., phase-change criterion, number of obtained reinforcers in each phase), but also to differences in the number of responses per sequence or, alternatively, to the universe of possible sequences. Bachá-Méndez et al. investigated sequences with two responses (i.e., four possible sequences), Sánchez-Carrasco and Nieto and Reed and Morgan used sequences of three responses (i.e., eight possible sequences) and Villas-Bôas used sequences of four responses (i.e., 16 possible sequences). Therefore, one may assume that the higher the number of responses per sequence (and, thus, the number of possible sequences), less likely the resurgence of the target sequence. In the present study we have directly investigated this variable by comparing sequences with three and with five responses.

A second variable may have contributed to the different results found by Villas-Bôas (2006), Bachá-Méndez et al. (2007), Sánchez-Carrasco and Nieto (2005) and Reed and Morgan (2006), namely, the probability of emission of each possible sequence. When reinforcers are delivered according to a continuous reinforcement schedule (CRF) regardless of the sequence topography it is observed, for example, that LLLL and RRRR sequences are the most likely among the 16 possible sequences, and that the RLLL sequence is more likely than the LLLR one (Neuringer, Deiss & Olson, 2000). This differential frequency is likely to reflect the degree of difficulty in learning those sequences. For the RLLL sequence, the organism learns that the first response is to be emitted in an operandum and the others in the other. For the LLLR sequence, however, the organism must identify the exact moment (after three responses) of switching between operandum (Neuringer, 1993). Considering that in a universe of possible sequences some are more likely than others, and that this variable has not been isolated in the studies described herein, this study evaluated the contribution of a target-sequence probability (high and low) to the resurgence of this sequence.

Resurgence is typically observed in a context that induces behavioral variability, i.e., a context of extinction. Exceptions are the studies by Lieving and Lattal (2003, Experiment 4) and by Jarmolowicz and Lattal (2014). Even in these cases, during the Testing Phase reinforcers were presented aperiodically and, thus, the contingency comprised extinction periods. Behavioral variability, however, can result directly from contingencies of reinforcers. In other words, different sequences can be generated when reinforcers are contingent to the emission of (a) sequences other than those previously reinforced (Page & Neuringer, 1985) or (b) sequences with low frequency and not so recent (Souza & Abreu-Rodrigues, 2006). Considering this, the third goal of the present study was to investigate if the explicit reinforcement of variation in the emission of other sequences (i.e., other than those reinforced in the Training and Elimination phases), compared with the extinction-induced variation, would differentially affect the resurgence of S1.

Finally, this study investigated generalization among species of the results found in studies of response-sequence resurgence. Up to now, the resurgence of sequences had only been investigated with rats (Bachá-Méndez et al., 2007; Reed & Morgan, 2006; Sánchez-Carrasco & Nieto, 2005; Villas-Bôas, 2006). This study, however, focused on human organisms.

In sum, this work comprised two experiments that investigated the resurgence of sequences of responses. Experiment 1 investigated if the resurgence of a sequence with five responses would be differentially affected by its probability of occurrence (high or low) and by the context during the Testing Phase (extinction-induced variability and directly-reinforced variability). Experiment 2 shared the same goals of Experiment 1, but the behavioral unit was a sequence with three responses.

Experiment 1

Experiment 1 had two goals: (1) to evaluate if the (high or low) probability of a five-response sequence interferes on its resurgence; and, (2) to verify if the resurgence of this sequence is differentially affected by the experimental conditions (extinction and operant variation) in the Testing Phase.

Method

Participants. Twenty undergraduate students of both sexes and from different undergraduate courses participated in this experiment. All participants read and signed the Informed Consent Term before starting the experimental session. After the session, students were granted points in courses of the Departamento de Processos Psicológicos Básicos of the Instituto de Psicologia, for their participation. The project was approved by the Ethics Committee in Research with Human Beings of the Faculdade de Ciências da Saúde of the Universidade de Brasília (Protocol number: 045/12).

Environment and Equipment. The experiment was conducted in a 2.30 m x 1.80 m room, with two tables and two chairs, in addition to a PC and a matrix printer on one of the tables. A software developed in Visual Basic 6® was used to present the experimental events and record data.

Procedure. One single instruction was presented on the computer screen in the beginning of the session:

This is an experiment on learning. Your task is to build sequences of five responses. For such, you must use the F and J keys of the keyboard. Every time you press a letter, a colored circle will be shown on the screen, so you can track where you are in the sequence.

There are 32 possible sequences. For each correct sequence you will receive 10 points and for every 100 points you will receive a token to participate in a raffle by the end of the experiment. The computer will show the total number of points received and the number of tokens you will receive.

When you are ready to start, click on OK.

When the participant clicked on OK the screen with the instruction was replaced by a black screen with a point counter (left bottom corner of the screen), a token counter (right bottom corner of the screen), and five circles with white edge and black background, horizontally placed on the upper and central parts of the screen. The task consisted of emitting sequences of five responses distributed on the F and J keys of the keyboard. There were 32 possible sequences. At every trial, as the participant pressed the F and J keys, the circles were automatically filled with red and yellow colors, respectively. If the emitted sequence met the reinforcement criterion in effect, the correct feedback was displayed. This feedback consisted in the display of a screen with a Smiley face and the phrase “You scored 10 points” below it. After 2 s, the screen with black circles with white edges was displayed again, but now the point and the token counters showed the points and tokens the participant had received, respectively. The screen also displayed, under the circles, a pyramid showing the Smiley faces accumulated up to then. Points and tokens were accumulated across trials until the end of the experimental session. The Smiley faces were accumulated until they formed a pyramid with 10 faces, corresponding to the emission of 10 correct sequences and the receiving of 100 points. When the pyramid was completed, the Smiley faces disappeared, a token was added to the token counter and when the next correct sequence was emitted a new pyramid then started. If the emitted sequence did not meet the reinforcement criterion in effect, a timeout (TO) was displayed. This TO consisted in the display of a black screen for 2 s. A new trial was initiated after each feedback or TO.

Participants were distributed in four experimental groups (High-Ext, High-Var, Low-Ext and Low-Var) and exposed to the Training, Elimination and Testing phases, all in effect in the same session that lasted about 1 h.

In a pilot study that comprised 20 participants, the target sequences to be used in each of the first two phases of the experiment were selected. To select these sequences, participants were exposed to 100 trials. A contingency of variation (threshold 0.03) superimposed to a VI 1-min schedule (see Neuringer et al., 2000) was in effect across trials. Therefore, after 1 min, on average, the emission of a sequence produced the correct feedback only if its frequency was lower or equal to 3% of the total of sequences emitted up to the moment. Otherwise, a TO was presented. The sequences emitted by all participants were ordered according to their frequency of occurrence. The most frequent sequence (FJFJF) was identified as the sequence with “high probability” and the less frequent one (JFJJF) as the sequence with “low probability”. Finally, the frequency of the sequence with “intermediate probability” (JJFJJ) was close to the arithmetic mean of the frequencies of the high- and low-probability sequences. The three sequences (high, low and intermediate probabilities) were then used in the experimental phases described below.

Training Phase. In this phase, the CRF schedule was in effect for a single sequence (S1). For the High-Ext and High-Var groups S1 was the sequence with high probability, while for the Low-Ext and Low-Var groups S1 was the sequence with low probability. Therefore, the emission of S1 (high or low probability) was always followed by a correct feedback, while the remaining 31 sequences were always followed by TO.

Elimination Phase. In this phase the sequence with intermediate probability (S2) was reinforced according to the CRF schedule for all groups. Thus, every time this sequence was emitted, a correct feedback was shown. The emission of the remaining sequences, including S1, was followed by TO.

Testing Phase. In this phase the contingency in effect varied among groups. For the High-Ext and Low-Ext groups an extinction contingency was in effect, and for the High-Var and Low-Var groups a variation contingency was in effect. When the extinction contingency was in effect, all possible sequences were followed by TO, including S1 and S2. When the variation contingency was in effect S1 and S2 were followed by TO, while the remaining 30 sequences produced a correct feedback if they met the reinforcement contingency. This contingency consisted of a variation criterion superimposed to a VI 1-min schedule. That is, all sequences (30) differing from S1 and S2 were reinforced only if emitted after 1 min, on average, and if they simultaneously met the threshold criterion of 0.50. Sequences that did not meet the reinforcement contingency were followed by TO.

The Training and Elimination phases ended when the participant emitted S1 (and S2) 50 times, or completed 200 trials, whichever came first. If this criterion was not met, the experiment was ended. The Testing Phase, in turn, was concluded after 100 trials. The changes of phases were not signaled.

Results

Figure 1 shows the percentage of occurrence of S1 and S2 in blocks of 10 trials in the Training, Elimination and Testing phases for each participant in each group. During the Training Phase (TR) all participants learned S1 despite its probability. The S1 learning criterion was met after 83 trials, on average, by the High-Ext and High-Var groups, and after 142 and 124 trials, on average, by the Low-Ext and Low-Var groups, respectively. In the Elimination Phase (EL) the percentage of occurrence of S1 decreased across trials for all participants, such that in the last four or five blocks this sequence was not emitted. It was observed also that S2 was learned in about 82 and 68 trials by the High-Ext and High-Var groups, respectively, and 91 and 80 trials by the Low-Ext and Low-Var groups, respectively. In the Testing Phase (TT) S2 occurred for all participants, but with lower frequency when compared with the previous phase. In the context of extinction S1 reappeared for all participants despite of its probability (high or low). In the context of variation, when S1 was a high-probability sequence it reappeared for all participants, but when it was a low-probability sequence it reappeared for one single participant (B-V 111).

Figure 1
Percentage of occurrence of sequences 1 (S1) and 2 (S2) during the Training (TR), Elimination (EL) and Testing (TT) phases for each participant in each group of Experiment 1.

Table 1 presents the rates of reinforcers and sequences in the Training Phase for each participant of each group. In this phase all participants emitted S1 50 times before completing 200 trials, i.e., they received 50 reinforcers. The average rates of reinforcers were higher for the groups with S1 with high probability (16 and 14 reinforcers per minute) than for groups with S1 with low probability (seven and nine reinforcers per minute). The average rates of sequences were similar to the four groups (between 18 and 21 responses per minute).

Table 1
Selected S1, reinforcers rate (reinforcements per minute) and sequence rate (sequences per minute) to each participant during the Training Phase of Experiments 1and 2. Position of S1 frequency in relation to the frequency of each of the 32 possible sequences and U value to each participant during the Testing Phase of Experiments 1 and 2.

Experiment 1							Experiment 2
5 Responses per Sequence - Same sequence							3 Responses per Sequence - Individual Sequence
Groups	Part	Seq	R rate	Seq Rate	S1 Position	U Value	Groups	Part	Seq	R rate	Seq Rate	S1 Position		U Value
												TT 1	TT 2	TT 1	TT 2
High-Ext	A-E 201	FJFJF	25	28	4th	0.93	High-Ext	A-E 02	JFJ	59	70	4th		0.99
	A-E 202	FJFJF	6	10	10th	0.95		A-E 04	JJJ	20	32	1st		0.97
	A-E 203	FJFJF	37	40	1st	0.90		A-E 06	FJF	15	18	2nd		0.97
	A-E 204	FJFJF	12	23	10th	0.96		A-E 07	JJJ	18	23	2nd		0.96
	A-E 206	FJFJF	1	3	16th	0.96		A-E 11	FJF	24	35	3rd		0.94
	M (SD)		16 (13)	21 (13)	8.2 (5.8)	0.94 (0.02)		M (SD)		27 (16)	36 (18)	2.4 (1.1)		0.96 (0.02)
High-Var	A-V 101	FJFJF	10	14	7th	0.70	High-Var-Ext	A-V-E 03	JFJ	41	50	4th	1st	0.74	0.91
	A-V 103	FJFJF	23	25	13th	0.68		A-V-E 19	FFJ	27	35	7th	7th	0.73	0.92
	A-V 104	FJFJF	6	14	24th	0.83		A-V-E 31	JFJ	20	22	7th	2nd	0.90	0.97
	A-V 105	FJFJF	7	18	6th	0.79		A-V-E 33	JJJ	20	32	5th	2nd	0.82	0.99
	A-V 109	FJFJF	20	22	9th	0.88		A-V-E 35	JJF	30	33	5th	2nd	0.86	0.97
	M (SD)		13 (7)	19 (4)	11.8 (7.3)	0.78 (0.09)		M (SD)		27 (8)	34 (8)	5.6 (1.3)	2.8	0.81 (0.07)	0.95 (0.04)
Low-Ext	B-E 214	JFJJF	15	20	4th	0.95	Low-Ext	B-E 08	JFF	26	37	1st		0.99
	B-E 215	JFJJF	4	14	6th	1.00		B-E 15	JFF	16	26	7th		0.91
	B-E 217	JFJJF	4	13	18th	0.94		B-E 20	JFF	21	76	2nd		0.98
	B-E 222	JFJJF	8	24	10th	0.98		B-E 21	JFF	19	63	8th		0.96
	B-E 231	JFJJF	6	21	3rd	0.96		B-E 26	FFJ	16	20	5th		0.96
	M (SD)		7 (4)	18 (4)	8.2 (6.1)	0.96 (0.02)		M (SD)		20 (4)	45 (22)	4.6 (3.0)		0.96 (0.03)
Low-Var	B-V 108	JFJJF	6	19	No	0.59	Low-Var-Ext	B-V-E 32	FJF	44	52	6th	3rd	0.82	0.98
	B-V 110	JFJJF	21	26	No	0.65		B-V-E 34	FJJ	10	20	7th	3rd	0.79	0.91
	B-V 111	JFJJF	4	11	12th	0.70		B-V-E 38	JJF	33	40	7th	3rd	0.91	0.99
	B-V 114	JFJJF	6	16	No	0.87		B-V-E 39	JJJ	30	45	7th	8th	0.86	0.93
	B-V 115	JFJJF	8	17	No	0.51		B-V-E 40	FFJ	22	34	6th	4th	0.87	0.99
	M (SD)		9 (6)	18 (5)	2.4 (5.4)	0.66 (0.14)		M (SD)		28 (11)	38 (10)	6.6 (0.5)	4.2	0.85 (0.05)	0.96 (0.04)

• Note: Part = participant, Seq = sequence, R = reinforcement, M = mean, SD = standard deviation, ‘No’ = S1 did not appear.

Table 1 also shows the position of S1 frequency relative to the frequency of the other sequences for each participant in each group during the Testing Phase. For seven of the 10 participants in the High-Ext and High-Var groups S1 with high probability was one of the 10 most frequent ones (exceptions were participants A-E 206, A-V 103 and A-V 104), regardless of the context in effect being extinction or variation. S1 with low probability (Low-Ext and Low-Var groups) was also one of the 10 most frequent sequences to four of the five participants exposed to extinction (except for participant B-E 217), but was emitted by only one of the five participants exposed to variation (B-V 111).

Table 1 also shows the U value. This measure indicates the level of variability of all sequences emitted in the Testing Phase and was obtained through the following formula:

where RFi corresponds to the relative frequency of each possible sequence and n is the number of possible sequences. The U value ranges from 0 to 1. Values equal to 1 mean that all possible sequences were emitted with equal frequencies (maximum variability) and values equal to 0 mean that one single sequence was emitted (maximum repetition). Despite the S1 probability extinction tended to generate higher mean U values (above 0.9) than those produced by the variation contingency (below 0.8).

Figure 2 summarizes the Testing Phase results, showing the average frequency of S1with high and low probability in the contexts of extinction and variation. As observed in Figure 1 and Table 1, the reappearance of S1 was more frequent during extinction than during variation. Moreover, for the participants exposed to extinction the reappearance of S1 was not affected by its probability. However, for participants exposed to operant variant, S1 with high probability reappeared more frequently than S1 with low probability.

Figure 2
Mean S1 frequency when extinction (white bars) and variation (black bars) were in effect in the Testing Phase of Experiment 1.

Discussion

The results of Experiment 1 suggested that: (1) the high-probability sequence was learned as promptly as the intermediate-probability sequence, and both were learned faster than the low-probability sequence; (2) the reappearance of S1 with high probability was more frequent than that of S1 with low probability, mainly in the context of variation; and, (3) the reappearance of S1 was greater in the context of extinction than in the context of variation.

Both the higher readiness to learn S1 with high probability and S2 (with intermediate probability) compared to S1 with low probability, and the more frequent reappearance of S1 with high probability than S1 with low probability may be attributed to the intrasequence discriminative control, i.e., the control exerted by a response over the next response of a sequence (Abreu-Rodrigues, Hanna, de Melo Cruz, Matos & Delabrida, 2004). S1 with high probability (FJFJF) and the sequence with intermediate probability (JJFJJ) involved regular alternation between the F and J keys, while S1 with low probability (JFJJF) required alternations at variable points of the sequence. In the FJFJF sequence, for example, each response was always followed by the alternative response, while in the JFJJF sequence a response could be followed by the alternative response or by itself. If we consider that greater discriminative control is observed when the antecedent stimulus is a reliable predictor of the reinforcer (e.g., Blough, 1975), then the discriminative control exerted by a response on the next one may be more promptly developed in sequences with high and intermediate probability, resulting in quicker learning of these sequences.

If highly probable sequences are also those with more accurate intrasequence discriminative control, and considering that this control contributes to maintain sequence of responses as behavioral units (Reed, Schachtman & Hall, 1991), then sequences with greater discriminative control (high probability) would tend to reappear more frequently (or to be hardly extinguished) than those with lower discriminative control (low probability). This analysis is in line with demonstrations that when sequence reinforcement results in the formation of integrated behavioral units (with strong discriminative control), changes in the schedules of reinforcement do not alter the topography of those sequences. In the study of Reed et al., for example, the use of discriminative stimuli facilitated the learning of the target sequence, and promoted the maintenance of this sequence when the contingencies of reinforcement were changed.

The reinforcer rates, higher for groups exposed to S1 with high probability than for those exposed to S1 with low probability, suggest another source of control. However, attributing the more frequent reappearance of S1 with high probability to the higher rate of reinforces for that sequence demands care, since the contribution of the rate of reinforcers to resurgence is a controversial issue in literature. Some authors (e.g., Craig & Shahan, 2016; da Silva, Maxwell & Lattal, 2008) advocate that the rate of reinforcers is not a reliable predictor of resurgence or, at least, as reliable as the rate of responses (here, the rate of sequences did not vary between groups or contexts). Other authors (e.g., Podlesnik & Shahan, 2010), however, suggest a direct relation between rate of reinforcers and magnitude of resurgence.

The effects of high or low probability upon the reappearance of S1 were higher in the context of operant variation than in that of extinction. Three aspects should be considered in the analysis of the role played by the context. First, extinction induced the emission of sequences that differed from S1 and S2. The induction of sequences that have not been previously trained is in line with experimental results of previous studies (e.g., Souza, Abreu-Rodrigues & Baumann, 2010; Villas-Bôas, 2006) that evidenced that when a sequence no longer produces reinforcers, the organisms tend to emit alternative sequences. In other words, they tend to present varied responses, at least in the beginning of extinction.

Secondly, extinction induced higher levels of variability than those found in the context of operant variation. This seems to be counterproductive considering that variability in the emission of sequences was not a requirement for the delivery of reinforcers in the context of extinction, but it was in the context of variation. This may have occurred because the variation criterion used in the context of operant variation was not very strict, allowing eventual repetition of sequences. In other words, since the threshold value was 0.50, the same sequence could be repeated in 50% of the trials. Therefore, this contingency, despite reinforcing variation, also allowed sequence repetition (Abreu-Rodrigues, 2005; Schwartz, 1982). Under extinction, in turn, the repetition of a sequence was never reinforced, which induced the emission of a different sequence after each sequence.

The third point refers to the more frequent reappearance of S1 in the context of extinction when compared to that of operant variation. The reappearance of S1 in the context of extinction is in line with the results of other studies in which the extinction of sequences of responses was programmed (e.g., Bachá-Méndez et al., 2007; Reed & Morgan, 2006; Sánchez-Carrasco & Nieto, 2005). The less frequent reappearance of S1 in the context of operant variation, in turn, probably occurred because the inclusion of reinforcers for alternative sequences superimposed the effects of the S1 reinforcement history. This analysis is consistent with the proposal by Lieving and Lattal (2003). These authors argue that resurgence should be understood in a continuum with high reinforcement to alternative responses (or sequences) on one extreme, and low or no reinforcement to these responses (or sequences) on the other extreme. Therefore, the presence of reinforcers for alternative responses (or sequences), as happened in this context of operant variation, increases the resurgence of target response (sequence). On the other hand, the absence of reinforcers for alternative responses (sequences) increases the likelihood of resurgence.

In brief, the results of Experiment 1 showed that S1 reappeared mainly when it was a sequence with high probability and when there were no reinforcers for alternative sequences. However, the S1 reappearance was not more frequent than that of other sequences (except of participant A-E 203), a result that may have occurred due to the number of responses by sequence (or, alternatively, the number of possible sequences). The following experiment investigated this possibility.

Experiment 2

Although S1 reappeared in Experiment 1, this cannot be considered as an unchallengeable evidence of resurgence because S1 was not more frequent than the other sequences. Therefore, three aspects of the procedure for Experiment 2 were changed in Experiment 1, in order to promote the resurgence of S1.

First, the number of responses per sequence was shortened (from five to three). This reduction was done because studies with rats showed resurgences with sequences of two (Bachá-Méndez et al., 2007) and three (Sanchéz-Carrasco & Nieto, 2005; Reed & Morgan, 2006) responses, but not with sequences of four responses (Villas-Bôas, 2006). Secondly, in Experiment 1 the target sequences were the same for all participants in a given group. Although sequences with high and intermediate probability were learned quicker than the sequence with low probability, there was some variability among participants. To investigate if the criterion of sequence selection affected the reappearance of S1, Experiment 2 used specific sequences for each participant. That is, the sequence with high (or low) probability corresponded to the sequence emitted more (or less) frequently by each participant in a pre-experimental phase. Thirdly, in Experiment 1 the effects of the contexts of extinction and operant variation were evaluated through between-group manipulations. In Experiment 2, in turn, these effects were additionally investigated through intrasubject manipulations, i.e., two groups of participants were exposed to two testing phases (operant variation and extinction) rather than just one (variation or extinction). Therefore, we could evaluate if the higher reappearance of S1 during extinction than during variation, as observed in Experiment 1, also occurs when we introduce the context of extinction after the context of variation.

Method

Participants. Twenty undergraduate students participated in this experiment. The other details were the same as those for Experiment 1.

Environment and Equipment. The data collection room and the equipment used were identical to those in Experiment 1.

Procedure

Participants were distributed in four groups (High-Ext, High-Var-Ext, Low-Ext and Low-Var-Ext) and exposed to four phases (Selection of the Target Sequence, Training, Elimination and Testing) in one single session.

In the beginning of the session, participants were given general instructions that differed from those in Experiment 1 only regarding the number of responses by sequence (three) and the maximum number of possible sequences (eight). After reading the instruction, the participant was exposed to a task identical to that described in Experiment 1.

Phase of Selection of the Target Sequences. In this phase the participant should emit sequences of three responses, according to a VI 1-min schedule in 50 trials. Therefore, sequences emitted at every 1 min, on average, were followed by the correct feedback. In this phase, conversely to what was done in the pilot study of Experiment 1, there was no variation criterion to prevent participants from initiating the experiment with a history of reinforcement for variation. Then, the experimenter selected the target sequences based on the absolute frequency of all possible sequences, as described in Experiment 1, except for one difference: instead of ordering aggregate frequencies of sequences emitted by all participants, the experimenter ordered the frequencies of sequences emitted by each individual participant. Therefore, a sequence with high (more frequent), intermediate (medium frequency) and low (less frequent) probability was obtained for each participant.

Training Phase. Identical to that of Experiment 1, i.e., only S1 generated reinforcers that were delivered according to the CRF schedule.

Elimination Phase. Identical to Experiment 1, i.e., reinforcers were delivered whenever, and only when, S2 was emitted.

Testing Phase 1. For the High-Ext and Low-Ext groups this phase was identical to the Testing Phase of the same groups in Experiment 1. That is, no reinforcer was delivered after any of the eight possible sequences. For the High-Var-Ext and Low-Var-Ext groups this phase was identical to that of the Testing Phase of the High-Var and Low-Var groups in Experiment 1. Namely, a contingency of variation (threshold 0.5) superimposed on a VI 1-min schedule was in effect for all sequences, except for S1 and S2 which were always followed by TO.

Testing Phase 2. Only the High-Var-Ext and Low-Var-Ext groups were exposed to this phase, which was identical to the Testing Phase of the High-Ext and Low-Ext groups in Experiment 1. In other words, no reinforcers were delivered.

The remaining aspects of the procedure were identical to those described in Experiment 1.

Results

The percentage of occurrence of S1 and S2 in blocks of 10 trials in the Training, Elimination and Testing phases for each participant of each group is shown in Figure 3. In the Training Phase all participants learned S1 regardless of which sequence was defined as S1 for each participant, and of its probability (high or low). For the High-Ext and High-Ext-Var groups the learning criterion was fulfilled after 66 trials, on average. The remaining groups needed a higher average number of trials to learn the sequence with low probability, mainly participants B-E 20 (184 trials) and B-E 2 (167 trials). The other participants fulfilled the criterion after 73 trials, on average. In the Elimination Phase the percentage of occurrence of S1 in the first block of 10 trials decreased in at least 50% (except for participants A-E 07, A-E 11 and A-V-E 19). This sequence was extinguished in the final blocks of this phase. Still in this phase the learning of S2 (the only sequence eligible for reinforcement), just like S1, differed among participants. For all groups, S2 was learned after a similar average number of trials (between 54 and 63), close to those of S1 with high probability in the Training Phase. In Testing Phase 1 and 2 all participants emitted S2, but less frequently than in the previous phase. In addition, S1 reappeared for all participants. The magnitude of this reappearance was greater during extinction than during variation regardless of the presentation order of the extinction context. However, this figure does not clearly show the effects of S1 probability.

Figure 3
Percentage of occurrence of sequences 1 (S1) and 2 (S2) during the Training (TR), Elimination (EL) and Testing (TT) phases for each participant in each group of Experiment 2.

As indicated in Table 1, in the Training Phase all participants received 50 reinforcers. The mean rate of reinforcers for the Low-Ext group (20 reinforcers per minute) was lower than that for the remaining groups (between 27 and 28 reinforcers per minute) and mean rate of sequences (45 sequences per minute) higher than the other groups (between 34 and 38 sequences per minute). Moreover, the rate of reinforcers and sequences in both phases were higher in Experiment 2 than in Experiment 1.

Table 1 shows that S1 with high probability was one of the four most frequent sequences for nine of the 10 participants in the context of extinction (except for participant A-V 19). On the other hand, in the context of variation S1 was among the four less frequent sequences for four of five participants (except for participant A-V 03). S1 with low probability was among the four most frequent sequences for six of the 10 participants during extinction, but one of the three less frequent sequences for all participants exposed to variation.

Table 1 also presents the U value. Just like in Experiment 1, the mean U value was higher during extinction (above 0.95) than during variation (equal to or lower than 0.85), both for the groups exposed to S1 with high probability and for those exposed to S1 with low probability.

Figure 4 shows the average frequency by opportunity of S1 with high and low probability in the contexts of extinction and variation in Experiment 1 (left panel) and Experiment 2 (central panel). It also shows the results of an additional group (right panel) that will be discussed in the General Discussion section. Since the universe of possible sequences changed when S1 had five or three responses, there were fewer opportunities for S1 to be emitted in the 100 trials of the Testing Phase of Experiment 1 than in those of Experiment 2. Therefore, a higher number of S1 occurrences in Experiment 2 would not necessarily indicate greater magnitude of S1 resurgence, since S1 could have occurred more frequently only because there were more opportunities for that. To avoid this problem of interpreting the results, the mean frequency of S1 was divided by the number of possible sequences in Experiments 1 and 2 (32 and 8, respectively). The average frequencies of both extinction situations did not differ between the High-Ext (2.1) and High-Var-Ext (2.15) groups, and between the Low-Ext (1.17) and Low-Var-Ext (1.17) groups of Experiment 2. That is why the figure only presents the mean for the first two and last two groups.

Figure 4
Mean S1 frequency by opportunity when extinction (white bars) and variation (black bars) were in effect in the Testing Phase of Experiment 1 (left panel), Experiment 2 (center panel) and additional group (right panel).

The reappearance of S1 was more frequent when this sequence comprised three responses (Experiment 2) than when it comprised five responses (Experiment 1). In addition, just as in Experiment 1, S1 reappeared more frequently in the context of extinction than in that of variation during Experiment 2. Regarding the level of difficulty, when the sequence had five responses S1 with high probability reappeared more frequently than S1 with low probability, but only during variation. When the sequence had three responses, S1 with high probability was also more frequent than S1 with low probability, mainly during extinction.

Discussion

Some results of Experiment 2 replicated those of Experiment 1: (1) S1 with low probability was learned more slowly and reappeared less frequently than S1 with high probability; and, (2) the reappearance of S1 was more frequent in the context of extinction than of variation. These results were replicated, despite the change in the number of sequences and in the procedure used to select the target sequences. Conversely to what happened in Experiment 1, the difference between the reappearance of S1 with high and low probability was higher in the context of extinction than in that of variation. Finally, S1 with three responses (Experiment 2) reappeared more often than S1 with five responses (Experiment 1).

The faster learning and the more frequent reappearance of S1 cannot be attributed exclusively to the intrasequence discriminative control, as happened in Experiment 1, because the same sequence may have been selected as high probability for a participant and as low probability for another participant (see participants A-V-E 35 and B-V-E 38, Table 1). Moreover, the intrasequence control is likely to have been developed similarly between sequences. That is because, since the sequence had only three responses, there was some regularity between the responses of the eight possible sequences.

Differences in learning and reappearance of S1 with high and low probability, however, may have been caused by variables present in the Phase of Selection of the Target Sequences. Although there is no record of the number of reinforcers contingent to each sequence in this phase, S1 with high probability, being the most frequent sequence, is likely to have received more reinforcers than S1 with low probability (the less frequent). This may have favored the reappearance of S1 in the Testing Phase (Winterbauer, Lucke & Bouton, 2013).

The less frequent reappearance of S1 in the context of variation than in that of extinction, as observed in Experiment 2, may be explained based on the reinforcement of the nontarget sequences, as was done in Experiment 1. However, in both contexts the magnitude of S1 reappearance was greater in Experiment 2 when compared with Experiment 1 (Figure 4). This result may be attributed to the number of responses in each sequence. In Experiment 2 the sequence had three responses, and in Experiment 1 five, which suggests that the lower the number of responses per sequence, the greater the reappearance of S1. Perhaps, reinforcing a sequence with lower number of responses, as previously discussed, makes the intrasequence discriminative control more accurate than in the case of a sequence with a higher number of responses. This control, in turn, may facilitate the reappearance of shorter sequences. The literature on numerosity suggests that the discriminative control accuracy varies directly with the number of responses in the sequence (e.g., Machado & Rodrigues, 2007; Mechner, 1958).

The comparison between Experiments 1 and 2 shows that in Experiment 1 S1 with high probability reappeared more frequently than S1 with low probability, but only in the context of variation. In Experiment 2, however, the more frequent reappearance of S1 with high probability was mainly in the context of extinction. This difference between experiments may have resulted from the relation between the number of possible sequences and the degree of discriminability of the absence of reinforcers for S1. The induced variability (context of extinction) and the intermittent reinforcement of nontarget sequences (context of variation) may have hindered the discriminability of the absence of reinforcers for S1, mainly when there were 30 (Experiment 1), in comparison to six (Experiment 2) nontarget sequences. Considering this, in the context of variation the occurrence of reinforcers for nontarget sequences is likely to have contributed to the more frequent reappearance of S1 with high probability, especially when the discriminability of S1extinction was low (Experiment 1). In the context of extinction, in turn, since no sequence produced reinforcers, it would be expected that the greater the discriminability of S1 extinction (Experiment 2), the lower the likelihood of reappearance of S1 with low probability. In fact, this situation was observed.

General discussion

This study evaluated the probability (high vs. low) that a sequence of responses determined by that sequence frequency in the pilot study (Experiment 1) and in the Phase of Selection of the Target Sequences (Experiment 2), and the contexts (extinction vs. variation) in effect in the Testing Phase, characterized by the total absence of programmed reinforcers and by the delivery of reinforcers contingent to a variability criterion, respectively, would affect the resurgence of that sequence. We found that: (1) sequences with high probability reappeared more frequently than sequences with low probability; (2) the context of extinction promoted more frequent reappearance of the target sequence than the context of variation; and, (3) the reappearance of the target sequence varied inversely with the number of responses per sequence.

The first two results were broadly discussed in Experiments 1 and 2, but the third result requires some additional considerations. Although studies about the role of reinforcers rate on the reappearance of a response have inconsistent results (e.g., Craig & Shahan, 2016; Podlesnik & Shahan, 2010; Silva et al., 2008), both in Experiment 1 and Experiment 2 of the present study the reappearance of S1 was correlated to higher rate of reinforcers. Moreover, the number of responses per sequence or, more precisely, the intrasequence discriminative control, which is stronger in sequences with lower number of responses (Abreu-Rodrigues et al., 2004), should also be considered. As noted in Experiment 1, by favoring the maintenance of sequence integrity this control, which is characterized by the predictive effect of each response upon the next response in the sequence, may also have promoted its reappearance (Reed et al., 1991).

To investigate the isolated contribution of the number of reinforcers and responses per sequence to the reappearance of S1, an extra manipulation was performed after Experiment 2. An additional group, made up by four participants, was exposed to the same experimental conditions as the High-Ext group in Experiment 1 (sequences with five responses), but the procedure to select sequences was individualized (i.e., it was identical to that of Experiment 2). As a result, the number of reinforcers for S1 in Experiment 2 groups and in the additional group was similar and higher than that in Experiment 1 groups. S1 with high probability reappeared for all participants in the additional group, and this reappearance was higher than in Experiment 1, but lower than in Experiment 2 (Figure 4). The comparison of the results of the additional group with those of the same group in Experiment 1 (both with five-response sequences, but the first with higher number of reinforcers for S1) provides evidence that the number of reinforcers should not be neglected. On the other hand, when the results of the additional group are compared with those of the same group in Experiment 2 (both with similar number of reinforcers but different number of responses per sequence), the number of responses per sequence appears as a relevant controlling variable. In brief, the results of the additional group suggest that the two variables (number of reinforcers and number of responses per sequences) affected the reappearance of S1.

It is worth highlighting that in this study the reappearance of S1 was not conceptualized as resurgence. This because although S1 has reappeared, its frequency of occurrence was not higher than that for the other sequences in the Testing Phase. This is an important point because extinction may induce the emission not only of responses that were previously trained, but also of new responses (Catania, 1998/2006). The possibility of inducing responses other than the trained ones raises some questions: (1) If the occurrence of S1 in the Testing Phase is less frequent than the occurrence of other sequences, it is legitimate to characterize the reappearance of S1 as resurgence?; (2) When nontarget sequences are not measured, it is feasible to affirm that the reappearance of S1 is resurgence?; (3) If resurgence is the reappearance of a previously reinforced sequence that is later extinguished, which role would the reinforcement of S1 play if sequences never reinforced can be induced in greater magnitude than S1 itself? These questions require methodological refinement in resurgence studies (i.e., the measurement of control responses, as was done by some authors such as Bachá-Méndez et al., 2007; Bishop, 2008; Bruzek et al., 2009; Reed & Morgan, 2006; Sanchéz-Carrasco & Nieto, 2005; Sweeney & Shahan, 2013; Villas-Bôas, 2006) as well as a conceptual review of resurgence.

The measurement of control sequences allows separating the effect of extinction-induced variability from the effect of the history of reinforcement (Bishop, 2008). If, for example, the frequency of the control sequence is higher than or equal to that of S1, then the interruption of reinforcers for S2 explains the reappearance of S1 (as well as the occurrence of the control sequence). On the other hand, if the S1 frequency is higher than that of the control sequence, its reappearance should be attributed to the experimental history of reinforcement. Therefore, if the concept of resurgence refers to the reappearance of a previously reinforced response (R1) in face of the extinction of another response (R2), and if such extinction induces variation in the emission of responses, then, as a result of the previous reinforcement, the occurrence of R1 should be more frequent than the occurrence of other responses.

Future studies should carry out manipulations that may increase the probability of S1 resurgence. The emission of sequences of responses distributed on two keys entails several topographically similar sequences, potentially hindering the resurgence of the target sequence. Therefore, an alternative would be to add differential discriminative stimuli in each experimental phase (see Kincaid, Lattal & Spence, 2015). This idea seems to be plausible, since literature shows evidence of resurgence of key pecking (Doughty, da Silva & Lattal, 2007) and of temporal pattern of responses (Cançado & Lattal, 2011) that were trained in an experimental situation with exteroceptive stimuli.

Referências

• Abreu-Rodrigues, J. (2005). Variabilidade comportamental. Em J. Abreu-Rodrigues & M. R. Ribeiro (Orgs.), Análise do comportamento: pesquisa, teoria e aplicação (pp. 189-210). Porto Alegre: Artmed.
• Abreu-Rodrigues, J., Hanna, E. S., de Mello Cruz, A. P., Matos, R., & Delabrida, Z. (2004). Differential effects of midazolam and pentylenetetrazole on behavioral repetition and variation. Behavioural Pharmacology, 15, 535-543.
• Alessandri, J., Lattal, K. A., & Cançado, C. R. X. (2015). The recurrence of negatively reinforced responding of humans. Journal of the Experimental Analysis of Behavior, 104, 211-222.
• Bachá-Méndez, G., Reid, A. K., & Mendoza-Soylovna, A. (2007). Resurgence of integrated behavioral units. Journal of the Experimental Analysis of Behavior, 87, 5-24.
• Bishop, M. R. (2008). Resurgence of operant variability. Tese de Doutorado, University of Nevada, Reno.
• Blough, P. M. (1975). Wavelength generalization and discrimination in the pigeon. Perception & Psychophysics, 12, 342-348.
• Bruzek, J. L., Thompson, R. H., & Peters, L. C. (2009). Resurgence of infant caregiving responses. Journal of the Experimental Analysis of Behavior, 92, 327-343.
• Cançado, C. R. X., & Lattal, A. K. (2011). Resurgence of temporal patterns of responding. Journal of the Experimental Analysis of Behavior, 95, 271-287.
• Catania, A. C. (2006). Aprendizagem: comportamento, linguagem e cognição (D. G. Souza, Org. Trad.). Porto Alegre: Artmed . (Trabalho original publicado em 1998).
• Cleland, B. S., Foster, T. M., & Temple, W. (2000). Resurgence: The role of extinction. Behavioural Processes, 52, 117-129.
• Craig, A. R., & Shahan, T. A. (2016). Behavioral momentum theory fails to account for the effects of reinforcement rate on resurgence. Journal of the Experimental Analysis of Behavior, 105, 375-392.
• da Silva, S. P., Maxwell, M. E., & Lattal, A. K. (2008). Concurrent resurgence and behavioral history. Journal of the Experimental Analysis of Behavior, 90, 313-331.
• Doughty, A. H., da Silva, S. P., & Lattal, K. A. (2007). Differential resurgence and response elimination. Behavioural Processes, 75, 115-128.
• Epstein, R. (1985). Extinction-induced resurgence: Preliminary investigations and possible applications. The Psychological Record, 35, 143-153.
• Jarmolowicz, D. P., & Lattal, K. A. (2014). Resurgence under delayed reinforcement. Psychological Record, 64, 189-193.
• Kincaid, S. L., Lattal, K. A. & Spence, J. (2015). Super-resurgence: ABA renewal increases resurgence. Behavioural Processes, 115, 70-73.
• Lieving, G. A., & Lattal, K. A. (2003). Recency, repeatability, and reinforce retrenchment: An experimental analysis of resurgence. Journal of the Experimental Analysis of Behavior, 80, 217-233.
• Machado, A., & Rodrigues, P. (2007). The differentiation of response numerosity in the pigeon. Journal of the Experimental Analysis of Behavior, 88, 153-178.
• Mechner, F. (1958). Probability relations within response sequences under ratio reinforcement. Journal of the Experimental Analysis of Behavior, 1, 109-122.
• Neuringer, A. (1993). Reinforced variation and selection. Animal Learning & Behavior, 21, 83-91.
• Neuringer, A., Deiss, C., & Olson, G. (2000). Reinforced variability and operant learning. Journal of Experimental Psychology: Animal Behavior Processes, 26, 98-111.
• Page, S., & Neuringer, A. (1985). Variability is an operant? Journal of Experimental Psychology: Animal Behavior, 11, 429-452.
• Podlesnik, C. A., & Shahan, T. A. (2010). Extinction, relapse, and behavioral momentum. Behavioural Processes, 84, 400-411.
• Reed, P., & Morgan, T. A. (2006). Resurgence of response sequences during extinction in rats shows a primacy effect. Journal of the Experimental Analysis of Behavior, 86, 307-315.
• Reed, P., Schachtman, T. R., & Hall, G. (1991). Effect of signaled reinforcement on the formation of behavioral units. Journal of Experimental Psychology: Animal Behavior Processes, 17, 475-485.
• Sánchez-Carrasco, L. S., & Nieto, J. (2005). Resurgence of three response sequences in rats. Revista Mexicana de Análisisde la Conducta, 31, 215-226.
• Schwartz, B. (1982). Reinforcement-induced behavioral stereotypy: How not to teach people to discover rules. Journal of Experimental Psychology: General, 111, 23-59.
• Souza, A. S., & Abreu-Rodrigues, J. (2006). Discriminative properties of vary and repeat contingencies. Behavioural Processes, 85, 116-125.
• Souza, A. S., Abreu-Rodrigues, J., & Baumann, A. A. (2010). History effects on induced and operant variability. Learning & Behavior, 38, 426-437.
• Sweeney, M. M., & Shahan, T. A. (2013). Effects of high, low, and thinning rates of alternative reinforcement on response elimination and resurgence. Journal of the Experimental Analysis of Behavior, 100, 1-15.
• Sweeney, M. M., & Shahan, T. A. (2016). Resurgence of target responding does not exceed increases in inactive responding in a forced-choice alternative reinforcement procedure in humans. Behavioural Processes, 124, 80-92.
• Villas-Bôas, A. (2006). Efeitos do procedimento de extinção sobre o responder em ratos: sequências de respostas como operante. Dissertação de Mestrado, Universidade de São Paulo, São Paulo.
• Winterbauer, N. E., & Bouton, M. E. (2010). Mechanisms of resurgence of an extinguished instrumental behavior. Journal of Experimental Psychology, 36, 343-353.
• Winterbauer, N. E., Lucke, S., & Bouton, M. E. (2013). Some factors modulating the strength of resurgence after extinction of an instrumental behavior. Learning and Motivation, 44, 60-71.

Publication Dates

• Publication in this collection
  2016

History

• Received
  07 Sept 2016
• Accepted
  06 Nov 2016