Excerpt from
Cardinal, R.N. (2001). Neuropsychology of reinforcement processes in the rat. Unpublished PhD thesis, University of Cambridge.
Copyright © Rudolf Cardinal 2001. All rights reserved.
Systematic technique for assessment of preference for delayed reinforcement
Eight identical operant conditioning chambers were used (30 × 24 × 30 cm; Med Instruments Inc, Georgia, Vermont, USA; Modular Test Cage model ENV-007CT). Each chamber was fitted with a 2.8-W overhead house light and two retractable levers with a 2.8-W stimulus light above each lever. Between the two levers was an alcove fitted with a traylight (60 mcd diffused green LED; RS Components Ltd, UK), an infrared photodiode to detect head entry (nosepokes), and a tray into which could be delivered 45-mg food pellets (Rodent Diet Formula P, Noyes, Lancaster, NH). The chambers were enclosed within sound-attenuating boxes fitted with fans to provide air circulation. The apparatus was controlled by software written by R.N. Cardinal in C++ using the Whisker control system (Cardinal & Aitken, 2002) [altered from original thesis, for which I used an antiquated control system].
Training. Subjects were first trained under an FR1 schedule to a criterion of 50 presses in 30 min, first for the left lever and then for the right. They were then trained on a simplified version of the full task. The session began with the levers retracted and the operant chamber in darkness. Every 40 s, a trial began with illumination of the houselight and the traylight. The subject was required to make a nosepoke response within 10 s, or the current trial was aborted and the chamber returned to darkness. If the subject nosepoked within this time limit, the traylight was extinguished and a single lever presented. If the rat failed to respond on the lever within 10 s, the lever was retracted and the chamber darkened, but if it responded, a single pellet was delivered immediately and the traylight was illuminated until the rat collected the pellet (or a 10-s collection time limit elapsed, whereupon the chamber was darkened). In the Houselight condition, the houselight was left on until 6 s after the food had been collected; in the Cue and No Cue conditions it was switched off at the moment the lever was pressed.
Choice of signalled and unsignalled delayed reinforcement. Subjects may choose between a small, immediate reward and a large, delayed reward. In the 'Cue' condition, a stimulus light is illuminated during the delay to reinforcement; this stimulus is therefore paired with the large reinforcer and may become a conditioned reinforcer.
In every pair of trials, the left lever was presented once and the right lever once, though the order within the pair of trials was random. Rats were trained to a criterion of 60 successful trials in one hour (the maximum possible with a 40-s period being 90).
Behavioural procedure. The task was based on Evenden and Ryan's (1996) procedure and is illustrated in figure…. Aside from the use of an extra signal during the delay, the present task differs from that of Evenden and Ryan in a number of ways; in particular, the subjects were required to initiate the trials and choose a lever within a limited time, and a forced-choice trial on each lever was given at the start of each block of choice trials at a given delay. Additionally, in their procedure the houselight was always on, whereas in the present studies the houselight was extinguished during the intertrial interval (ITI), making it an informative stimulus (in that food was delivered when the houselight was on, but never when it was off). Finally, subjects were not given exposure to the large reinforcer before delays were introduced into the task.
The session began in darkness with the levers retracted; this was designated the intertrial state. Trials began at 100-s intervals. Each trial began with the illumination of the houselight and the traylight. The rat was required to make a nosepoke response, ensuring that it was centrally located at the start of the trial (latency to poke was designated the initiation latency). If the rat did not respond within 10 s of the start of the trial, the operant chamber was reset to the intertrial state until the next trial began and the trial was scored as an omission. If the rat was already nosepoking when the trial began, the next stage followed immediately.
.jpg)
Schematic of the task. On the right-hand side, the format of a single trial is shown. This diagram shows in detail the Houselight condition, in which the houselight remains on from the start of the trial until 6 s after the subject has collected the reward. On the left-hand side, the differences between the three lighting conditions are illustrated. In the No Cue condition, the houselight is switched off at the moment of choice. In the Cue condition, the houselight is similarly switched off when the subject responds on a lever, but a stimulus light is illuminated during the delay that precedes delivery of the large reinforcer.
Upon a successful nosepoke, the traylight was extinguished and one or both levers were extended. One lever was designated the Delayed lever, the other the Immediate lever (counterbalanced left/right). The latency to choose a lever was recorded. (If the rat did not respond within 10 s of lever presentation, the chamber was reset to the intertrial state until the next trial and the trial was scored as an omission.) When a lever was chosen, both levers were retracted. Choice of the Immediate lever caused the immediate delivery of one pellet; choice of the Delayed lever caused the delivery of 4 pellets following a delay. In the Cue condition, the houselight was switched off at the moment of choice and a stimulus light above the chosen lever switched on for the duration of the delay. In the No Cue condition, the stimulus light was not switched on. In the Houselight condition, the houselight remained on instead. These three conditions are illustrated in figure….
Following any delay, the stimulus light was switched off, the traylight was switched on and the reinforcer for that lever was delivered. Multiple pellets were delivered 0.5 s apart. If the rat collected the pellets before the next trial began, then the time from delivery of the first pellet until a nosepoke occurred was recorded as the collection latency. The traylight was switched off, and in the Houselight condition the houselight remained on for another 6 s (eating time). In other conditions there was no houselight illumination during this time. If the rat did not collect the food within 10 s of its delivery, the operant chamber entered the intertrial state, though collection latencies were still recorded up to the start of the next trial. The chamber was then in the intertrial state and remained so until the next trial. There was no mechanism to remove uneaten pellets, but failure to collect the reward was an extremely rare event (see Results).
The delay was varied systematically across the session. A session consisted of 5 blocks, each comprising two trials on which only one lever was presented (one trial for each lever, in randomized order) followed by ten free-choice trials. Delays for each block were 0, 10, 20, 40 and 60 s respectively. As trials began every 100 s, the total session length was 100 minutes; subjects received one session per day.