Temporal control in chained fixed-ratio, fixed-interval schedules

BEHAVIOR/SYSTEMS/COGNITION

Temporal control in chained fixed-ratio, fixed-interval schedules

João Claudio Todorov^I; Kalliu Carvalho Couto^II; Lucas Couto de Carvalho^II

^IUniversidade de Brasília, Brasília, DF, Brazil

^IICentro Universitário IESB, Brasília, DF, Brazil

^{Correspondence}

ABSTRACT

Four rats were subjected to chained fixed-ratio (FR), fixed-interval (FI) schedules of reinforcement (chain FR 5 FI). A FR schedule at one lever produced a discriminative stimulus (i.e., light) associated with an FI schedule of primary reinforcement (water) at the second response lever. The FR schedule was kept constant, whereas the FI length was changed from 10 to 60 s under five different experimental conditions. Increases in the FI length resulted in increases in pre-ratio pauses, but pauses in the FI tended to be a constant percentage of FI length. Data from this experiment indicate that pre-ratio pauses are also a function of the interreinforcement interval (IRI). Data from three experiments with chained FR 5 FI 60-s schedules indicate that pausing in the FI component of chained FR FI schedules with the FI as the second component of the chain may tend to disappear as the IRI duration increases.

Keywords: temporal control, pre-ratio pauses, FI pauses, interreinforcement interval, rats.

Introduction

Research that involves fixed-interval (FI) schedules of reinforcement began serendipitously (Skinner, 1956) but opened a new area of investigation in experimental psychology. Even critics of B.F. Skinner recognize the importance of schedules of reinforcement for the study of learning and motivation (Staddon, 2001). Originally referred to as periodic schedules (e.g., Keller & Schoenfeld, 1950), FI schedules specify a minimum, fixed time between the last reinforcement or some other event in the environment and the next opportunity for reinforcement (e.g., Ferster & Skinner, 1957; Catania, 1991; Moreira & Medeiros, 2007). Thus, the FI may begin and end with primary reinforcement (e.g., Cançado & Lattal, 2011), begin with another event and finish with primary reinforcement, or begin with some event and finish with conditioned reinforcement (e.g., Hanson, Campbell, & Witoslawski, 1962; Mechner, Guevrekian, & Mechner, 1963; de Souza & Todorov, 1975; Todorov & Teixeira-Sobrinho, 2009; Jimenez-Gomez & Shahan, 2012).

Two response patterns emerge in simple FI schedules (e.g., Ferster & Skinner, 1957; Cumming & Schoenfeld, 1958; Shull, 1970a). In one pattern, after an initial pause, response rates increase gradually (scallop) during the interval until the next reinforcement. In the other pattern, response rates increase abruptly after the initial pause (i.e., break-and-run). In both patterns, pauses represent ~50–80% of the size of the interval before response acceleration (Schneider, 1969; Machado, Malheiro, & Erlhagen, 2009). Pausing in an FI has been attributed to the fact that the period after reinforcement is a signal for the absence of response reinforcement (Skinner, 1938; Ferster & Skinner, 1957; Catania, 1991).

In a simple fixed-ratio (FR) schedule of reinforcement, a fixed number of n responses is necessary to receive reinforcement. Pre-ratio pauses and work time are features of the pattern of responding in this schedule; both measures increase with increases in the size of the FR requirement. When the pre-pause and work times are increased, the interreinforcement interval (IRI) is also extended. Therefore, at least three variables can influence the pre-pause length in this schedule: FR requirement, interreinforcement interval, and work time (e.g., Neuringer & Schneider, 1968; Shull, 1970a; Crossman, Heaps, Nunes, & Alferink, 1974).

In chained FR FI schedules, completion of the response requirement in the first component of the chain produces a stimulus change that functions as conditioned reinforcement for FR responding and a discriminative stimulus for the beginning of the FI (i.e., the second component of the chain). If the time to the next reinforcement is the controlling variable in FI schedules, then FI pauses should remain constant with changes in the FR schedule. However, previous studies showed that pauses in the second component decrease with increases in the FR requirement (de Souza & Todorov, 1975; Todorov & Teixeira-Sobrinho, 2009; Todorov, Carvalho, Couto, da Cruz, & Cunha, 2012). Temporal control in chained schedules appears to be a function of the IRI and not a function of the fixed time to the next opportunity for reinforcement signaled by a discriminative stimulus.

The present study was designed to determine the involvement of the IRI in the temporal control of behavior in a chained FR FI schedule. Previous studies manipulated the IRI by increasing the size of the FR schedules. In contrast, the present study manipulated the IRI by increasing the size of the FI length while keeping the FR requirement constant.

Methods

Subjects

Four naive, male Wistar rats, aged 6 months at the beginning of the experiment (described as rats 2, 4, 8, and 13) were used. The rats were born and maintained in the vivarium of the Centro Universitário IESB and individually housed in polycarbonate cages (30 × 30 × 50 cm) under a 12 h/12 h light/dark cycle with constant temperature (22 ± 2ºC) and relative humidity (55%). Food was available at all times, and access to water was restricted for 48 h before each experimental session.

Apparatus

Four MedAssociates Modular Test Chambers (MedAssociates ENV-008; SN: 3318) for rats were used. The chambers had two standard response levers and access to water controlled by an electromechanical device. The water access was centrally located between the response levers. During reinforcement, a dipper presented 0.06 ml of water for 3 s. A house light was located on the wall opposite the wall with the response levers, and two lights could be turned on or off above each lever. All events within the experimental chamber were scheduled and recorded using a computer compatible with IBM-PC interface DIG-700P1 and Windows MedPC software (SOF-735). Data were recorded using Schedule Manager software with Visual Basic and developed especially for the present study.

Procedure

After shaping lever-pressing through the differential reinforcement of successive approximations of that response class, the subjects were gradually exposed to chained FR 5 FI 10 s schedules. The number of sessions and hours required to shape FR and FI lever-press responding for each rat was not recorded. Under this first condition, five responses on the right lever (FR 5) turned on a light above the left lever and initiated a 10-s period (FI 10 s). The first response after 10 s from the onset of the light turned off the light above the lever and resulted in the presentation of water for 3 s. Sessions occurred three times per week (Monday, Wednesday, and Friday) (rats 2 and 4) or Tuesday, Thursday, and Saturday (rats 8 and 13). The rats were exposed to each experimental condition for a minimum of nine sessions. The stability criterion required that the medians of FI pausing for each group in three sessions did not show any tendency. For each rat, the median for that group of nine sessions was representative of that experimental condition. In successive experimental conditions, the FI intervals were fixed at 15, 30, 45, and 60 s.

Data were recorded as post-reinforcement pauses (PRPs) or pre-ratio pauses in the FR schedule, interreinforcement intervals (IRIs), the number of reinforcements in the session, pauses in the FI schedules, and duration of the session. A PRP was defined as the length of time between the end of access to water and the first response in the FR schedule. A pause during the FI was defined as the length of time between the onset of the discriminative stimulus (i.e., light) and the first response in the presence of that stimulus.

Results and Discussion

Table 1 shows the number of sessions for each rat in each experimental condition. Figure 1 shows that pausing in chained FR 5 FI schedules tended to be a constant percentage of FI length in the range FI 10 s to FI 45 s, a result consistent with the literature (e.g., Ferster, & Skinner, 1957, p. 170; Schneider, 1969; Zeiler, & Powel, 1994). However, subjects differed with regard to the values that were used to calculate those percentages. Figure 1 also shows data from a chained FR 5 FI 60 s schedule from Todorov & Teixeira-Sobrinho (2009) and Todorov et al. (2012). Data from Rat 2 in the FI 60 s condition may suggest that pausing in the FI schedule in chained FR FI schedules, with the FI as the second component of the chain, may tend to disappear as the IRI duration increases.

A remarkable characteristic of Figure 1 is the intersubject variability that was possibly produced by exposure to the chained FR 5 FI 10 s schedule in the first experimental condition. Any overshooting of the FR requirement on one lever might result in reinforcement for the first response on the FI lever, a situation similar to concurrent superstition (e.g., Skinner, 1948; Herrnstein, 1961; Todorov, 1971; Todorov & Ramirez, 1981; Lejeune, Richelle, & Wearden, 2006). Such an effect would result in long pauses before responding on the FI schedule, which may reflect superstitious responding on the FR lever (i.e., accidental reinforcement). A related effect is found in single-key, response-initiated FI schedules (Shull, 1970b) when the initial FI length is short. Shull used sequential FI durations of 3.75, 7.5, 15, 30, and 60 s. Under such conditions, FI pausing never developed, and the response pattern (i.e., "breakand-run") was similar to ratio schedules, with PRPs increasing as the FI increased.

Figure 2 shows pre-ratio pauses (i.e., pauses before the FR) as a function of FI length in chained FR 5 FI schedules. Pauses before the FR systematically increased as the FI duration increased. Figure 2 also shows pre-ratio pauses in a chained FR 5 FI 60 s schedule from Todorov & Teixeira-Sobrinho (2009) and Todorov et al. (2012). Pauses in the FR schedule in these two previous studies increased with slight increases in the FR schedule. Post-reinforcement pauses showed larger increases than expected based on similar FR data (e.g., Powell, 1968; Crossman, Trapp, Bonem, & Bonem, 1985). The present and previous results (Todorov & Teixeira-Sobrinho, 2009; Todorov et al., 2012) may suggest that pausing in the FR schedule in chained FR FI schedules may tend to increase as the IRI duration increases.

In chained FR FI schedules, either with increases in FR requirement or increases in FI length, pauses tend to be concentrated after primary reinforcement and responding in the component that is closer to the next primary reinforcement. Crossman (1968) compared pauses between multiple and chained FR FR schedules. Pre-FR 10 pauses were shorter than pre-FR 100 pauses in multiple schedules. However, in chained FR 10 FR 100 schedules, Crossman found that pauses in the first component were longer than in the second component.

Data from 11 rats (five rats from Todorov & Teixeira-Sobrinho, 2009; four rats from Todorov et al., 2012; two rats from the present study) and two rats in Figure 1 show that pauses in FI schedules in the chained FR 5 FI 60 s schedule are shorter than in simple FI schedules. An important consideration is the different experimental manipulations in these studies. The present study manipulated the IRI by changing the FI schedule, and the previous studies manipulated the IRI by changing the FR schedule.

The present and previous results (Todorov & Teixeira-Sobrinho, 2009; Todorov et al., 2012) indicate that temporal control in FI or FR schedules may be influenced by contextual variables. At least four variables might influence the temporal control over FI pauses in chained FR FI schedules, including the size of the FR requirement, the discriminative stimulus, FI length, and IRI.

Competing general theories of timing behavior usually favor the isolation of time as the independent variable in a particular task such as the Scalar Expectancy Theory (Gibbon, 1977), Behavioral Theory of Timing (Killeen & Fetterman, 1988, 1993), Multiple-Time-Scale model (Staddon & Higa, 1999), and Learning-to-Time model (Machado, 1997; Machado & Arantes, 2006). The present study addressed a different task in which the elapsed times between events are part of the observed sources of multiple control over responding. One of the most studied procedures is the concurrent chained schedule with FI schedules in the terminal links, usually addressed by theories that are very different from those mentioned above (e.g., Grace, 1994; Grace & Nevin, 1997; Luco, 1990).

Acknowledgements

The authors would like to thank Centro Universitário IESB and Márcio Borges Moreira for the use of equipment and installations, João Vianney Severo for help with the software, Nayara de Souza Gois and two anonymous reviewers for suggestions that improved the manuscript, and Gisele Maria Rosa Sobrinho, Gleidson Gabriel da Cruz, and Claudia Octavia Ribeiro da Cunha for help conducting the experimental sessions. João Claudio Todorov thanks CNPq (Brazil) for a research scholarship.

Received 11 December 2012

Received in revised form 21 February 2013

Accepted 29 March 2013

Available online 27 June 2013

• Cançado, C. R. X., & Lattal, K. A. (2011). Resurgence of temporal patterns of responding. Journal of the Experimental Analysis of Behavior, 95, 271-287.
• Catania, A. C. (1991). Time as a variable in behavior analysis. In I. H. Iversen, & K. A. Lattal (Ed.), Experimental analysis of behavior (pp. 1-19). Amsterdam: Elsevier.
• Crossman, E. K. (1968). Pause relationship in multiple and chained fixed-ratio schedules. Journal of the Experimental Analysis of Behavior, 11, 117-126.
• Crossman, E. K., Heaps, R. S., Nunes, D. L., & Alferink, L. A. (1974). The effects of number of responses on pause length with temporal variables controlled. Journal of the Experimental Analysis of Behavior, 22, 115-120.
• Crossman, E. K., Trapp, N. L., Bonem, E. J., & Bonem, M. K. (1985). Temporal patterns of responding in small fixed-ratio schedules. Journal of the Experimental Analysis of Behavior, 43, 115-130.
• Cumming, W. W., & Schoenfeld, W. N. (1958). Behavior under extended exposure to a high-value fixed-interval reinforcement schedule. Journal of the Experimental Analysis of Behavior, 1, 245-263.
• de Souza, D. G., & Todorov, J. C. (1975, Outubro). Esquema encadeado razão fixa-intervalo fixo: Efeitos do tamanho da razão sobre a pausa pós-reforço. In V Reunião Anual de Psicologia, Sociedade de Psicologia de Ribeirão Preto, Ribeirão Preto, Brasil.
• Ferster, C. B., & Skinner, B. F. (1957). Schedules of reinforcement. New York: Appleton-Century-Crofts.
• Gibbon, J. (1977). Scalar expectancy theory and Weber's law in animal timing. Psychological Review, 84, 279-325.
• Grace, R. C. (1994). A contextual model of concurrent-chains choice. Journal of the Experimental Analysis of Behavior, 61, 113-129.
• Grace, R. C., & Nevin, J. A. (1997). On the relation between preference and resistance to change. Journal of the Experimental Analysis of Behavior, 67(1), 43-65.
• Hanson, H. M., Campbell, E. H., & Witoslawski, J. J. (1962). FI length and performance on an FI FR chain schedule of reinforcement. Journal of the Experimental Analysis of Behavior, 5, 331-333.
• Herrnstein, R. J. (1961). Relative and absolute strength of responses as a function of frequency of reinforcement. Journal of the Experimental Analisys of Behavior, 4, 267-272.
• Jimenez-Gomez, C., & Shahan, T. A. (2012). Concurrent-chains schedules as a method to study choice between alcohol-associated conditioned reinforcers. Journal of the Experimental Analysis of Behavior, 97, 71-83.
• Keller, F. S., & Schoenfeld, W. N. (1950). Principles of psychology: Systematic text in the science of behavior New York: AppletonCentury-Crofts.
• Killeen, P. R., & Fetterman, J. G. (1988). A behavioral theory of timing. Psychological Review, 95, 274-295.
• Killeen, P. R., & Fetterman, J. G. (1993). The behavioral theory of timing: Transition analyses. Journal of the Experimental Analysis of Behavior, 59, 411-422.
• Lejeune, H., Richelle, M., & Wearden, J. H. (2006). About Skinner and time: Behavior-analytic contributions to research on animal timing. Journal of the Experimental Analysis of Behavior, 85, 125-142.
• Luco, J. E. (1990). Matching, delay-reduction, and maximizing models for choice in concurrent-chains schedules. Journal of the Experimental Analysis of Behavior, 54(1), 53-67.
• Machado, A. (1997). Learning the temporal dynamics of behavior. Psychological Review, 104, 241-265.
• Machado, A., & Arantes, J. (2006). Further tests of the Scalar Expectancy Theory (SET) and the Learning-to-Time (LeT) model in a temporal bisection task. Behavioural Processes, 72, 195-206.
• Machado, A., Malheiro, M. T., & Erlhagen, W. (2009). Learning to time: A perspective. Journal of the Experimental Analysis of Behavior, 92, 423-458.
• Mechner, F., Guevrekian, L., & Mechner, V. (1963). A fixed interval schedule in which the interval is initiated by a response. Journal of the Experimental Analysis of Behavior, 6, 323-330.
• Moreira, M. B., & Medeiros, C. A. (2007). Princípios básicos de análise do comportamento Porto Alegre: Artmed.
• Neuringer, A. J., & Schneider, B. A. (1968). Separating the effects of interreinforcement time and number of interreinforcement responses. Journal of the Experimental Analisys of Behavior, 11, 661-667.
• Powell, R. W. (1968). The effect of small sequential changes in fixed-ratio size upon the post-reinforcement pause. Journal of the Experimental Analysis of Behavior, 11, 589-593.
• Schneider, B. A. (1969). A two-state analysis of fixed-interval responding in the pigeon. Journal of the Experimental Analysis of Behavior, 12, 677-687.
• Shull, R. L. (1970a). The response-reinforcement dependency in fixed-interval schedule of reinforcement. Journal of the Experimental Analysis of Behavior, 14, 55-60.
• Shull, R. L. (1970b). A response-initiated fixed-interval schedule of reinforcement. Journal of the Experimental Analysis of Behavior, 13, 13-15.
• Skinner, B. F. (1938). The behavior of organisms: An experimental analysis New York: Appleton-Century-Crofts.
• Skinner, B. F. (1948). 'Superstition' in the pigeon. Journal of Experimental Psychology, 38, 168-272.
• Skinner, B. F. (1956). A case history in scientific method. American Psychologist, 11, 221-233.
• Staddon, J. E. R. (2001). The new behaviorism: Mind, mechanism, and society Philadelphia: Psychology Press.
• Staddon, J. E. R., & Higa, J. J. (1999). Time and memory: Towards a pacemaker-free theory of interval timing. Journal of the Experimental Analysis of Behavior, 71(2), 215-251.
• Todorov, J. C. (1971). Concurrent performances: Effect of punishment contingent on the switching response. Journal of the Experimental Analisys of Behavior, 16, 51-62.
• Todorov, J. C., Carvalho, L. C., Couto, K. C., da Cruz, G. G., & Cunha, C. O. R. (2012). Fixed-interval pause duration in chained fixed-ratio, fixed-interval schedules. Psychology, & Neuroscience, 5, 91-96.
• Todorov, J. C., & Ramirez, M. A. R. (1981). Delay of reinforcement for responses which end pauses: Effects on response. Revista Mexicana de Análisis de la Conducta, 7, 135-139.
• Todorov, J. C., & Teixeira-Sobrinho, J. P. (2009). Pausas em esquemas encadeados razão fixa, intervalo fixo: será o tempo senhor da razão? Revista Brasileira de Análise do Comportamento, 5, 43-48.

Publication Dates

• Publication in this collection
  02 Oct 2013
• Date of issue
  June 2013

History

• Received
  11 Dec 2012
• Accepted
  29 Mar 2013
• Reviewed
  21 Feb 2013