Summary
Subjects are offered a choice between two reinforcers, which can differ in their magnitude, probability, and delay.
Purpose
Choice with delayed and/or probabilistic reinforcement (discrete-trial task).
Originally based on Evenden JL, Ryan CN (1996). The pharmacology of impulsive behaviour in rats: the effects of drugs on response choice with varying delays of reinforcement. Psychopharmacology 128: 161–170.
Sample publications using this form of task (PMID refers to PubMed ID at http://www.pubmed.com)
| • | Cardinal RN, Robbins TW, Everitt BJ (2000). The effects of d-amphetamine, chlordiazepoxide, alpha-flupenthixol and behavioural manipulations on choice of signalled and unsignalled delayed reinforcement in rats. Psychopharmacology 152: 362–375. PMID 11140328. |
| • | Cardinal RN, Pennicott DR, Sugathapala CL, Robbins TW, Everitt BJ (2001). Impulsive choice induced in rats by lesions of the nucleus accumbens core. Science 292: 2499–2501. PMID 11375482. |
| • | Cardinal RN, Cheung THC (2005). Nucleus accumbens core lesions retard instrumental learning and performance with delayed reinforcement in the rat. BMC Neuroscience 6: 9. PMID 15691387. |
| • | Cheung THC, Cardinal RN (2005). Hippocampal lesions facilitate instrumental learning with delayed reinforcement but induce impulsive choice in rats. BMC Neuroscience 6: 36. PMID 15892889. |
| • | Cardinal RN, Howes NJ (2005). Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards in rats. BMC Neuroscience 6: 37. PMID 15921529. |
The following articles illustrate a quantitative methodology (to be commended) to establish aspects of reinforcer delay/magnitude/probability sensitivity; this paradigm can also be accomplished with the present task.
| • | Kheramin S, Body S, Mobini S, Ho MY, Velázquez-Martinez DN, Bradshaw CM, Szabadi E, Deakin JF, Anderson IM (2004). Effects of quinolinic acid-induced lesions of the orbital prefrontal cortex on inter-temporal choice: a quantitative analysis. Psychopharmacology 165: 9-17. PMID 12474113. |
| • | Kheramin S, Body S, Ho M, Velazquez-Martinez DN, Bradshaw CM, Szabadi E, Deakin JF, Anderson IM (2003). Role of the orbital prefrontal cortex in choice between delayed and uncertain reinforcers: a quantitative analysis. Behavioural Processes 64: 239-250. PMID 14580695. |
| • | Kheramin S, Body S, Ho MY, Velazquez-Martinez DN, Bradshaw CM, Szabadi E, Deakin JF, Anderson IM (2004). Effects of orbital prefrontal cortex dopamine depletion on inter-temporal choice: a quantitative analysis. Psychopharmacology 175: 206-14. PMID 14991223. |
| • | Bezzina G, Cheung TH, Asgari K, Hampson CL, Body S, Bradshaw CM, Szabadi E, Deakin JF, Anderson IM (2007). Effects of quinolinic acid-induced lesions of the nucleus accumbens core on inter-temporal choice: a quantitative analysis. Psychopharmacology 195: 71-84. PMID 17659381. |
Touchscreen and operant chamber versions of this task
This program (part of MonkeyCantab) is a touchscreen-based task. A separate task (known simply as ImpulsiveChoice) exists to run this task in conventional lever-based operant chambers; see http://www.whiskercontrol.com.
Typical trial structure
A number of types of task are possible with this program. The trial structure of a typical delayed-reinforcement task, albeit using operant chambers rather than touchscreens, is shown below (taken from Cardinal et al. 2001):
Likewise, the trial structure of a typical probability-based task is shown below (taken from Cardinal & Howes 2005):
Parameters
The parameters dialogue box looks like this:
The options are as follows:
| • | The stimulus with varying parameters (stimulus B) is left/right/mobile. Whereas the usual operant chamber version of this task associates a lever's side (e.g. left/right) with a particular reinforcement outcome, this touchscreen task associates a visual stimulus with a particular outcome. Suppose a red circle is your chosen stimulus for option A (which might be a small, immediate reward) and a green square is your chosen stimulus for option B (which might be a large, variably delayed reward). Should the option B stimulus always be on the left, always on the right, or variably on the left and right? |
| • | ... if mobile, left/right side is drawn without replacement from list of size 2 x N. If you choose the "mobile" option, then specify this option. You choose a number; we will call it N. When you run the task, N copies of "left" are put into a hat, and N copies of "right" are also put in. The contents of the hat are shuffled. For each trial in turn, an option is then drawn out at random (and not replaced); this option determines the side of option B. When the hat is empty, the procedure is repeated. So if you specify N = 1, then each consecutive trial pair contains one "A on the right / B on the left" trial and one "A on the left / B on the right" trial. If you specify a very large N, the procedure becomes close to drawing truly at random. See Drawing Without Replacement for more details. |
| • | As a further complexity, forced-choice trials are, by default, arranged in A/B pairs (see below). The "B side" is kept constant across that pair. A new "B side" is picked for each pair as described above. In free-choice trials, a new B side is picked for each trial as described above. |
| • | "Left"/"right" locations. Define the Locations used for the left and right stimuli. |
| • | Option A (typically fixed). This section specifies the parameters for option A. |
| • | Stimulus/manipulandum. What stimulus will be shown for the subject to touch to obtain option A? Set the stimulus (see Choosing Stimuli For A Task). Alternatively, untick "Same stimulus for all A options" and then you can specify the stimulus on a per-block basis. |
| • | Reinforcer A. Define the reinforcer delivered for this option. |
| • | Option B (typically variable).This section specifies the parameters for option B. |
| • | Stimulus/manipulandum. As for option A. |
| • | Reinforcer B. Define the reinforcer delivered for this option. |
| • | Block structure. |
| • | Here you specify a list of blocks. Each block contains a delay and probability for each of the A and B options, and if you are setting stimuli on a per-block basis, those too. |
| • | Task structure. |
| • | Trial initiation: spontaneous/lever/stimulus/magazine. Should trials initiate themselves spontaneously? Should a lever response be required to start the trial? Or should the subject have to touch a central stimulus (which you can set) to initiate each trial? Or should the subject have to respond in the food magazine (optionally, illuminating the magazine light to indicate this?) |
| • | Number of forced-choice trials per block. Trials are grouped into blocks. Blocks typically start with a certain number of forced-choice trials; by this I mean that only one option (A or B) is offered in each trial. Specify an even number here; "A" trials and "B" trials are (by default) delivered in pairs, with the A/B order within each pair being random. |
| • | Number of free-choice trials per block. After the forced-choice trials, there are free-choice trials, where A and B are offered. Specify the number of free-choice trials in each block. Remember that the delay/probability of option B stays constant within a block, but varies across blocks. |
| • | Shuffle trials within a block? Optionally, instead of having the forced-choice trials followed by the free-choice trials, you can shuffle (randomize) the order of all trials within a block. |
| • | Shuffle across all trials? If ticked, all the trials will be shuffled together (across blocks). |
| • | Stimulus bridges delay. If selected, a stimulus will be displayed during the delay to the A and B reinforcers (assuming that the response is rewarded). Choose the stimuli to be displayed when A and B have been selected, and choose whether this stimulus should be displayed on the side of the manipulandum (i.e. the A stimulus will be displayed on the A side, etc.) or in the centre of the screen. |
| • | "Central" location. Define the Location used for the central stimulus. |
| • | Houselight bridges delay. Ordinarily in this task, the houselight is on when waiting for trial initiation, and when waiting for a choice, and off at all other times. If this is ticked, the houselight is left on during any delay, and during reinforcement delivery/collection/feeding. |
| • | Magazine light bridges delay. If ticked, the magazine light will stay on during the delay. |
| • | Initiation limited hold. If the subject does not initiate the trial within this time, the trial is abandoned. (Not applicable for the "spontaneous" method of trial initiation; see above.) |
| • | Choice limited hold. If the subject does not choose A or B within this time, the trial is abandoned. |
| • | Collection limited hold. If the subject does not collect the reinforcer within this time, the task returns to the intertrial state. (Note that typical MonkeyCantab equipment cannot detect pellet collection, only licks at the pump; however, behaviourally it is unlikely to matter if the task returns to the intertrial state whilst pellets are being collected/eaten. Obviously, collection latency data will only be available for pump licking.) |
| • | Reinforcer collection time. Once the reinforcer has been collected, the lights stay on for this length of time before the task returns to the intertrial state. |
| • | Trials begin every... (known as the trial period, in the physics sense). Other things being equal, trials should begin at fixed intervals in delay-of-reinforcement tasks so as to avoid the confound of a subject choosing a short-delay reinforcer often and thus doing better than a subject choosing a long-delay reinforcer. This task fixes the trial period (the interval at which trials begin) so as avoid this problem. You must specify a time that exceeds the maximum reinforcer delay, plus initiation/choice/etc. limited hold times. |
| • | The session time is shown, calculated from the information you have provided. |
| • | Repeat {free, forced}-choice trials in which an omission occurs. Optionally, you may repeat (exactly) trials on which the subject makes an omission (i.e. fails to initiate, or fails to respond). Such repetitions will PROLONG the overall session time. |
| • | Pellet pulse length. This option should be set to match your pellet dispenser; see the General Parameters. |
| • | Time between pellets. This option should be set to match your pellet dispenser (and to make the arrival of multiple pellets distinctive to the subject); see the General Parameters. |
| • | Quick configuration options. These buttons provide commonly used default settings for the assessment of delay or probability sensitivity; see the publications cited above for further discussion. |
When you add or edit a delay/probability setting for option B, you'll see this:
If you have configured the task for independent stimuli for each delay/probability pair, you can also set the stimulus at this point.