Fixed-interval pause duration in chained fixed-ratio, fixed-interval schedules

a12v5n1

Fixed-interval pause duration in chained fixed-ratio, fixed-interval schedules

João Claudio Todorov^I; Lucas Couto de Carvalho^II; Kalliu Carvalho Couto^II; Gleidson Gabriel da Cruz^III; Claudia Octavia Ribeiro da Cunha^II

^IUniversidade de Brasília, Brasília, DF, Brazil

^IIInstituto de Educação Superior de Brasília, Brasília, DF, Brazil

^IIIInstituto Walden 4, Brasília, DF, Brazil

^{Correspondence}

ABSTRACT

Ten rats were submitted to chained fixed-ratio (FR), fixed-interval (FI) schedules of reinforcement. A FR schedule at one lever produced a discriminative stimulus associated with a FI 60-s schedule of primary reinforcement (water) at the second response lever. In Experiment 1, the FI schedule was kept constant while the FR requirement was changed from one to seven responses under five different experimental conditions for five rats. Increases in the FR requirement resulted in increases in post-reinforcement pauses but also decreases in pauses in the FI schedule. Using another five rats, Experiment 2 tested the hypothesis that short pauses in the FI schedule result from the use of the chained schedules procedure. Baseline was a FI 80-s schedule. In the second condition, chained FR 1 FI 80-s schedules were programmed. The third condition was a return to baseline. In baselines 1 and 2, the FI pause was compatible with the literature but decreased considerably when a chained schedule was used. The present results support the hypothesis that the time between primary reinforcement presentations dominates the control of FI pauses over control by the onset of a discriminative stimulus.

Keywords: timing; chained schedules; fixed ratio; fixed interval; water; rats.

Introduction

A review published by Lejeune, Richelle, and Wearden (2006) calls our attention to the importance of the work with the fixed-interval (FI) schedules of Ferster and Skinner (1957) for the study of interval timing in animals (e.g., Staddon & Cerutti, 2003). Timing behavior or the temporal control of behavior is the capacity to adjust behavior to temporal regularities in the environment in the range of seconds to minutes (Machado, Malheiro, & Erlhagen, 2009). Time as part of a three-term reinforcement contingency has been used as a discriminative stimulus (e.g., Arantes & Machado, 2008), definition of responding (e.g., Bejarano & Hackenberg, 2007), and parameter of the reinforcing stimuli (e.g., Todorov, 1973; Alsop & Porritt, 2006), but time as a discriminative stimulus that defines FI schedules has been most studied. Reviews of temporal control in FI schedules have been published during the past 50 years (e.g., Felton & Lyon, 1966; Dews, 1970; Davison, 1974; Shull, 1979; Lowe & Wearden, 1981; Lejeune & Wearden, 1991; Aparício, Lopez & Nevin, 1995; Machado, 1997; Lejeune et al., 2006).

In FI schedules, the possibility of a reinforcing stimulus that follows a response depends on the time since the last reinforcement or since the presentation of a discriminative stimulus (e.g., Ferster & Skinner, 1957; Catania, 1984). The two procedures are named "fixed-interval schedules" either in single or complex schedules as concurrent chains (e.g., Killeen, 1970; Davison, 1976; Moore, 2009). However, typical studies of temporal control use FI schedules defined as the time since the last reinforcement. One of several possibilities of studying timing behavior in FI schedules when the time to the next reinforcement is signaled by some event other than primary reinforcement is chained schedules with FI as the second component (Ferster & Skinner, 1957). Pilot studies (de Souza & Todorov, 1975; Todorov & Teixeira-Sobrinho, 2009) showed an unexpected result in which pauses in the second component of chained fixed-ratio (FR) FI schedules were shorter than typical pauses in single FI schedules (e.g., Sherman, 1959; Schneider, 1969). Pausing appeared to be controlled by the inter-reinforcement interval (variable) and not by the fixed time to the next opportunity for reinforcement signaled by a discriminative stimulus.

Todorov and Teixeira-Sobrinho (2009) studied chained FR n FI 60-s schedules with the FI 60-s kept constant while the FR increased from one to eight responses and reported that pausing in the FI schedule decreased with increases in the FR requirement. Post-reinforcement pauses (PRPs) increased as the FR increased, systematically changing the interval between primary reinforcements. An experiment conducted by de Souza and Todorov (1975) used a similar procedure with rats and reported the effect on PRPs and the absence of pauses during the FI schedule. The present study was a replication and extension of these experiments that sought to understand how and why the discriminative stimulus that signals the constant FI duration does not control pausing in chained schedules while retaining its conditioned reinforcer ability to maintain FR responding.

Experiment 1

Methods

Subjects

Five naive, male Wistar rats aged 6 months at the beginning of the experiment (described as rats 1, 3, 5, 6, and 7) were used. Rats were born and maintained in the vivarium of the Instituto de Educação Superior de Brasilia and were housed individually in polycarbonate cages (30 × 30 × 50 cm) and maintained on a 12 h/12 h light/dark cycle with constant temperature (22 ± 2ºC) and relative humidity (55%). Food was available at all times, and access to water was restricted for 48 before each experimental session.

Apparatus

Five MedAssociates Modular Test Chambers (MedAssociates ENV-008 SN: 3318) for rats were used. The chambers had two standard response levers and access to water controlled by an electromechanical device. Access to water was located centrally between the response levers. During reinforcements, a dipper presented 0.06 ml of water for 3 s. A houselight was located on the wall opposite the wall with the response levers, and two lights could be turned on or off above each lever. All events within the experimental chamber were scheduled and recorded using a computer compatible with IBM-PC interface DIG-700P1 and Windows Med-PC software (SOF-735). Data were recorded using Schedule Manager software using Visual Basic and developed especially for the present study.

Procedure

After the shaping of lever-pressing through the differential reinforcement of successive approximations of that response class, subjects were gradually exposed to chained FR 1 FI 60-s schedules. The number of sessions and hours required to shape lever-press responding for each rat was not recorded. Under this first condition, a response on the right lever (FR 1) turned on a light above the left lever and initiated a 60-s period (FI 60 s). The first response after 60 s from the onset of the light turned off the light above the lever resulted in the presentation of water for 3 s. Sessions occurred three times per week, either on Monday, Wednesday, and Friday for some rats or Tuesday, Thursday, and Saturday for other rats. Rats were exposed to each experimental condition for a minimum of nine sessions. The stability criterion required that the medians of FI pausing for each group of three sessions did not show any tendency for the last nine sessions. For each rat, the median for that group of nine sessions was representative of that experimental condition. In successive experimental conditions, FR requirements were fixed at 2, 3, 5, 7, and 1.

Data were recorded as PRPs, pauses in the FI schedule, IRI, number of reinforcements in the session, and duration of the session. A PRP was defined as the length of time between the end of access to water and the first response in the FR schedule. Pause during the FI was defined as the length of time between the onset of the discriminative stimulus (light) and the first response in the presence of that stimulus.

Results and Discussion

Table 1 shows the number of sessions for each rat in each experimental condition. Figure 1 shows that, for four of five rats, pauses in the FI 60-s schedule of chained FR FI schedules decreased as the FR requirement increased (p = .0196, Kruskal-Wallis analysis of variance [ANOVA]), replicating the data of Todorov and Teixeira-Sobrinho (2009). Pauses in the first experimental condition (FR 1) were lower than the pauses reported in the literature (e.g., Schneider, 1969). As the response requirement increased in subsequent experimental conditions, the pauses systematically decreased with the exception of Rat 1.

Figure 2 shows that in the first return-to-baseline session (chained FR 1 FI 60-s schedules), the FI pauses were very close to those observed in the first experimental condition. Notably, data that refer to the return to baseline are from the first session of the second exposure to the chained FR 1 FI 60-s schedules. Return to baseline levels must have occurred during the beginning of the session because the data in Figure 2 are presented as medians of all FI pauses in that session.

Post-reinforcement pauses (i.e., pauses before responding in the FR schedule began) increased with increases in the FR requirement for all five rats (p = .049, Kruskal-Wallis ANOVA; Figure 3). The magnitude of the effect altered the IRI. The obtained IRI was the sum of the PRPs plus a minimum of 60 s of the FI schedule.

Figures 4 and 5 show how the increases in the IRI affected the pattern of responding in the chained schedules. Data are from Rat 7 in the experimental conditions when FR 1 and FR 7 were used.

Figure 4 shows short PRPs and typical responding in the FI component of the chained schedules. In Figure 5, with FR 7, PRPs were longer, and responding in the FI began almost immediately when the schedule was on and sometimes continued at a steady rate until reinforcement or showing positive acceleration (scallop).

Data from Rat 1 were atypical and are not included in the analysis of Figure 1. When the FR requirement was 1, FI pauses were short (as observed by de Souza & Todorov, 1975) and increased with increases in FR requirement, probably showing the effect of continuous exposure to the FI 60-s schedule. Thus, for four of five rats, the results of Experiment 1 generally replicated the data of Todorov and Teixeira-Sobrinho (2009). In chained FR FI schedules, PRPs and IRIs increased whereas pauses in the FI decreased as the FR requirement increased. If FI schedules, defined as the time interval between primary reinforcements, control behavior similarly to FI schedules defined by the onset of some other change in the environment, then the FI pause duration in chained FR FI schedules should change as the FR requirement changes. Fixed-interval pauses could increase with continued exposure to the same FI duration (60 s). However, even with an FR 1, pauses in the FI were lower than pauses typically found in single FI schedules (e.g., Schneider, 1969). In the return to baseline, with a change from FR 7 to FR 1, the first session in the second exposure to FR 1 showed pauses that were very similar to those obtained after stability in the first experimental condition.

Data from the cumulative records of Rat 7 showed how the discriminative control of light over the response lever may have been overtaken by other variables. FR 1 had visible pauses in the FI schedules followed by the positive acceleration of responding until primary reinforcement. Large FI pauses were absent with the FR 7, and scallops were sometimes substituted by continuous responding until reinforcement.

Overall, these results could be interpreted as the primacy of IRIs in the control of pausing and responding in FI schedules (Staddon, Wynne, & Higa, 1991; Innis, Mitchell, & Staddon, 1993), even when the FI duration is determined by events other than the last primary reinforcement. However, instead of or in addition to that, the use of two response levers to better define the chained schedules may have interfered with the discriminative control of light over responding during the FI schedule. One way to explain the differences found in FI performance when the pauses began with primary reinforcement or another event is to compare data from Experiment 1 with data obtained with single FI schedules. Experiment 2 used an A-B-A design to compare pauses in single FI schedules (condition A) with pauses in chained FR FI schedules (condition B).

Experiment 2

Methods

Subjects

Five naive, male Wistar rats aged 6 months at the beginning of the experiment (described as Rats 9, 10, 12, 14, and 15) were used. Rats were born and maintained in the vivarium of the Instituto de Educação Superior de Brasilia and were housed individually in polycarbonate cages (30 × 30 × 50 cm) maintained under a 12 h/12 h light/dark cycle with constant temperature (22 ± 2ºC) and relative humidity (55%). Food was available at all times, and access to water was restricted for 48 before each experimental session.

Equipment

Equipment was the same as in Experiment 1.

Procedure

After shaping lever-press responding as in Experiment 1, subjects were exposed to a single FI 80-s schedule of reinforcement associated with the response lever with a light on over it. Stability criterion was the same as in Experiment 1. When the criterion was met, subjects were exposed to chained FR 1 FI 80-s schedules. The light over the lever was off after the reinforcements. A response on the left lever (FR 1) turned on a light above the right lever and initiated an 80-s period (FI 80 s). The first response after 80 s from the onset of the light turned off the light above the bar resulted in the presentation of water for 3 s. One response turned the light on and initiated the FI 80 s. After the stability criterion was met, subjects were again exposed to a single FI 80-s schedule for the number of sessions necessary to reach the stability criterion.

Results and Discussion

Table 2 shows the number of sessions for each rat in each experimental condition. Similar to Experiment 1, the number of sessions required to reach the stability criterion was low, but Figure 6 shows that the effect was considerable. The change to chained FR 1 FI 80-s schedules had an almost immediate effect of shortening the pauses. Return to baseline under the third experimental condition showed pauses in a single FI 80-s schedule that were compatible with those observed in the first baseline.

Figure 7 shows typical responding in the FI 80-s schedule (i.e., scallops). In Figure 8, with a chained FR 1 FI 80-s schedule, responding in the FI began almost immediately when the schedule was on and sometimes continued at a steady rate until reinforcement (i.e., no scallop).

For all five subjects, pauses in the single FI schedule were consistent with the literature (Cumming & Schoenfeld, 1958; Sherman, 1959; Schneider, 1969; Staddon et al., 1991; Innis et al., 1993). Performance in the single FI 80-s schedule was characterized by no responses in the first 50–80% of the interval followed by response acceleration until primary reinforcement. Under the second experimental condition, chained FR 1 FI 80-s schedules resulted in a qualitative change in behavior. Pauses in the FI schedule decreased abruptly with durations similar to those in Experiment 1.

Short FI pauses in chained FR FI schedules cannot be explained by shortcomings of the experimental procedure. Despite the fact that the same name is given to three different schedules (i.e., FI beginning and ending with primary reinforcement), the best explanation appears to be that in the FI that began with a discriminative stimulus and FI that ended with a conditioned reinforcer/discriminative stimulus, the crucial variable was not the time to the next reinforcement marked by some event but rather the obtained IRI. In single FI schedules, the minimum IRI is controlled by the experimenter. In chained schedules, the minimum IRI depends on the characteristics of the schedule programmed as the first component. In the present study, the first component of the chained schedules was an FR schedule. Changes in the FR requirement indirectly resulted in changes in the stable relationship between the last reinforcement and presentation of the discriminative stimulus associated with the next reinforcement.

General Discussion

The results from Experiments 1 and 2 and results of de Souza and Todorov (1975) and Todorov and Teixeira-Sobrinho (2009) showed that FI schedules may be useful to study time discrimination when defined as the time between successive presentations of primary reinforcement (e.g., Machado et al., 2009) but not when the FI is the second link in chained schedules (Ferster & Skinner, 1957). In this case, time to the next reinforcement is signaled by the presentation of a discriminative stimulus associated with responding in the FI schedule and a conditioned reinforcer for the response that produces it. A new issue is why temporal control is different in single and chained schedules.

At least part of the explanation for the present results could be a variable suggested by Shull (1970), specifically the number of responses emitted before the last reinforcement. This hypothesis would lead to longer pauses in the FI schedule as the response requirement in the FR schedule increases. However, this is not the case. Responding in the FI schedule began early in the interval, and the pause in the FI schedule was inversely proportional to the FR requirement.

The results from an experiment in which the FI schedule was the first link of a chain and thus began with primary reinforcement and ended with a conditioned reinforcer also reveal discrepancy. Hanson, Campbell, and Witoslawski (1962) used chained FI FR schedules with rats, food as reinforcement, long FI durations (for rats), and extended training before beginning the experiment. Pauses expressed as percentages of interval length are expected to be constant (cf. Machado et al., 2009), but Hanson et al. (1962) reported that pauses expressed as percentages decreased sharply as the FI duration increased and FR remained constant.

Some possible experimental procedures may be used to further understand temporal control in FI schedules—a question that remains without a widely accepted answer (Wearden & Lejeune, 2006). Chained fixed-time, fixed-interval (FT FI) schedules have the advantage of controlling postreinforcement pauses and response requirements before the onset of the discriminative stimulus that signals the beginning of the FI schedule. Chained FT FI schedules establish that no responding is necessary to produce the onset of the FI schedule. Variations in the duration of the FT schedule produce changes in the IRI, independent of the animal's behavior. Another way to describe this procedure is that it is equivalent to multiple extinction FI schedules (Ferster & Skinner, 1957), in which responding is maintained by an FI schedule in one component and subjected to extinction in another component. Concurrent schedules with an FI as one of the components of the concurrent pair may be another useful procedure. In this case, pauses in the FI schedule depend on the reinforcement context.Todorov and Ferreira (1978) reported "Visual aspects of cumulative record [in concurrent FI 50-s, FI 129-s schedules] showed that in the period after a reinforcement in FI 50-s the subjects would switch to responding in FI 102-s; few switches to the longer FI schedules occurred in the end part of FI 50-s" (Todorov & Ferreira, 1978, p.108). The general conclusion was that responding in longer FI schedules could be partially seen as interim activity controlled by the short FI duration.

Acknowledgments

The authors would like to thank the three anonymous reviewers for suggestions that improved the manuscript. The authors also thank Centro Universitário IESB and Márcio Borges Moreira for the use of equipment and installations. Wanderley Akira Shiguti helped with the statistical analyses. João Vianney Severo helped with the software. Gisele Maria Rosa Sobrinho helped with conducting the experimental sessions. João Claudio Todorov thanks CNPq (Brazil) for a research scholarship.

Received 20 December 2011

received in revised form 5 March 2012

accepted 26 March 2012

Available online 29 June 2012

• Alsop, B., & Porritt, M. (2006). Discriminability and sensitivity to reinforcer magnitude in a detection task. Journal of the Experimental Analysis of Behavior, 85, 4156.
• Aparício, C. F., Lopez, F., & Nevin, J. A. (1995). The relation between postreinforcement pause and interreinforcement interval in conjunctive and chain fixed-ratio fixed-time schedules. Psychological Record, 45, 105125.
• Arantes, J., & Machado, A. (2008). Context effects in a temporal discrimination task: further tests of the Scalar Expectancy Theory and learning-to-time models. Journal of the Experimental Analysis of Behavior, 90, 3351.
• Bejarano, R., & Hackenberg, T.D. (2007). IRT-stimulus contingencies in chained schedules: Implications for the concept of conditioned reinforcement. Journal of the Experimental Analysis of Behavior, 88, 215227.
• Catania, A.C. (1984). Learning. 2nd edition. Englewood Cliffs, NJ: Prentice-Hall.
• Cumming, W. W., & Schoenfeld, W. N. (1958). Behavior under extended exposure to a high-value fixed-interval reinforcement schedule. Journal of the Experimental Analysis of Behavior, 1, 245263.
• Davison, M. C. (1974). A functional analysis of chained fixed-interval schedule performance. Journal of the Experimental Analysis of Behavior, 21, 323330.
• Davison, M. C. (1976). Preference for fixed-interval schedules: Effects of unequal initial links. Journal of the Experimental Analysis of Behavior, 25, 371376.
• Dews, P. B. (1970). The theory of fixed-interval responding. In W. N. Schoenfeld (Ed.), The theory of reinforcement schedules (pp. 4361). New York: Appleton-Century-Crofts
• de Souza, D. G. & Todorov, J. C. (1975). Esquema encadeado razão fixaintervalo fixo: efeitos do tamanho da razão sobre a pausa pós-reforço. In V Reunião Anual de Psicologia, Sociedade de Psicologia de Ribeirão Preto, Ribeirão Preto, SP. Programa e Resumos
• Felton, M., & Lyon, D. O. (1966). The postreinforcement pause.Journal of the Experimental Analysis of Behavior, 9, 131134.
• Ferster, C. B. & Skinner, B. F. (1957). Schedules of reinforcement. New York: Appleton-Century-Crofts
• Hanson, H. M., Campbell, E. H., & Witoslawski, J. J. (1962). FI length and performance of an FI FR chain schedule of reinforcement. Journal of the Experimental Analysis of Behavior, 5, 331333.
• Innis, N. K., Mitchell, S. K., & Staddon, J. E. R. (1993). Temporal control on interval schedules: What determines the postreinforcement pause? Journal of the Experimental Analysis of Behavior, 60, 293311.
• Killeen, P. (1970). Preference for fixed-interval schedules of reinforcement. Journal of the Experimental Analysis of Behavior, 14, 127131.
• Lejeune, H., & Wearden, J. H. (1991). The comparative psychology of fixed-interval performance: Some quantitative analyses. Learning and Motivation, 22, 84111.
• Lejeune, H., Richelle, M., & Wearden, J. H. (2006). About Skinner and time: behavior-analytic contributions to research on animal timing. Journal of the Experimental Analysis of Behavior, 85, 125142.
• Lowe, C. F., & Wearden, J. H. (1981). Weber's law and the fixed-interval post-reinforcement pause. Behavior Analysis Letters, 1, 2737.
• Machado, A. (1997). Learning the temporal dynamics of behavior. Psychological Review, 104, 241265.
• Machado, A., Malheiro, M. T., & Erlhagen, W. (2009). Learning to time: A perspective. Journal of the Experimental Analysis of Behavior, 92, 423458.
• Moore, J. (2009). Some effects of procedural variations on choice responding in concurrent chains. Journal of the Experimental Analysis of Behavior, 92, 345365.
• Schneider, B. A. (1969). A two-state analysis of fixed-interval responding in the pigeon. Journal of the Experimental Analysis of Behavior, 12, 677687.
• Sherman, J. G. (1959). The temporal distribution of responses on fixed interval schedules [unpublished thesis]. New York: Columbia University
• Shull, R. L. (1970). A response-initiated fixed-interval schedule of reinforcement. Journal of the Experimental Analysis of Behavior, 13, 1315.
• Shull, R. L. (1979). The post-reinforcement pause: Some implications for the correlational law of effect. In M. D. Zeiler& P. Harzem (Eds.), Reinforcement and the organization of behavior [series title: Advances in analysis of behavior, vol. 1 (pp. 193222). New York: Wiley
• Staddon, J. E. R., & Cerutti, D. T. (2003). Operant conditioning.Annual Review of Psychology, 54, 115144.
• Staddon, J. E. R., Wynne, C. D. L., & Higa, J. J. (1991). The role of timing in reinforcement schedule performance. Learning and Motivation, 22, 200225.
• Todorov, J. C. (1973). Interaction of frequency and magnitude of reinforcement on concurrent performances. Journal of the Experimental Analysis of Behavior, 19, 451458.
• Todorov, J. C., & Ferreira, M. C. C. (1978). Multiple and concurrent schedules of fixed-interval reinforcement. Revista Mexicana de Análisis de la Conducta, 4, 101109.
• Todorov, J. C., & Teixeira-Sobrinho, J. P. (2009). Pausas em esquemas encadeados razão fixa, intervalo fixo: será o tempo senhor da razão? Revista Brasileira de Análise do Comportamento, 5, 4348.
• Wearden, J. H., & Lejeune, H. (2006). "The stone which the builders rejected...": Delay of reinforcement and response rate on fixed-interval and related schedules. Behavioural Processes, 71, 7787.

Publication Dates

• Publication in this collection
  26 Oct 2012
• Date of issue
  June 2012

History

• Received
  20 Dec 2011
• Accepted
  26 Mar 2012
• Reviewed
  05 Mar 2012