Programming models for behavioural tasks

The technique you adopt can make task programming easy or difficult. Think through your task carefully before beginning.

Types of task and appropriate programming models

Most behavioural tasks that I've come across use one or more of the following styles of programming.

Free-operant tasks

Simple reinforcement schedules fall into this category. Take the example of a ratio schedule: whenever a lever is pressed, a counter is advanced, and whenever the counter reaches a certain number (the ratio requirement), reinforcement is delivered and the counter is reset.

In Whisker, where everything is driven by events, this is phenomenally easy to code. First, you ask the server to send you a message when a lever is pressed – perhaps BattleshipPotemkin. Part of your program will receive messages coming from the server. If a message begins with Event:, the rest of the message should be checked. If you had received Event: BattleshipPotemkin, then the rat has pressed a lever. Increment your counter variable and if appropriate, send commands to deliver reinforcement.

State models

An extension of this model is a state-based system. To be frank, every behavioural task can be programmed using this model, but it's worth considering explicitly. The program maintains a representation of what state the task is in. When it receives information about the subject's actions, its response is based on (1) what the subject did, but also (2) what state the task is in.

As a very simple example, and an extension of the ratio schedule, suppose you want to ignore lever-presses while reinforcement is being delivered. You would implicitly have defined two possible States of the Box: hanging around waiting and delivering reinforcement. When the lever-press message comes in and the box is in the delivering reinforcement state, nothing is done about it; if the box is in the hanging around state instead, the schedule counter is incremented. In this example, you would also set up timers to inform you when enough reinforcer had been delivered, and this would trigger the state to change from delivering reinforcement to hanging around. (The transition in the opposite direction obviously occurs when the rat fulfils the ratio.)

I have found that many novice programmers use state-based programming techniques without really realizing it. I think it is well worth representing the state of the box explicitly in your code. (There are often more states than you think!)

State models in complex tasks

Sometimes, of course, your task may be too complicated to represent with a single state chain. Perhaps you are training animals to respond under a discriminative stimulus. Different stimuli are presented and lever-pressing is measured during each stimulus. The stimuli are controlled on one progression (e.g. interstimulus interval → S1 → ISI → S2…), and you want to ignore lever-presses while your pump is running, so you implement waiting and delivering reinforcement states as well. It would be easiest to represent these states in two variables (maybe called StimulusSequenceState and LeverState). That way the transition from one state to the other is clearly defined for each variable (if a single variable was used for both, you'd have to work out which stimulus state to go back into when the reinforcer has been delivered, and so on). The response to a lever-press would then depend on the conjoint state of both variables – reinforcement would be delivered when the discriminative S+ was on and the pump was off.

Furthermore, states may be controlled on the basis of time (and therefore, in Whisker, timers). A random interval schedule sets up a 'ticker'; every second, reinforcement is or is not 'set up'. When the rat presses a lever, reinforcement is only delivered if it has been set up by the timer-based schedule. This is another example easily implemented if you think about the states involved.

Most complicated tasks are very easily represented using state models. For example, the notorious five-choice serial reaction time task has various states: when a stimulus is presented, for example (the Stimulus On state?), several things can happen: the rat can respond correctly or incorrectly, or it can fail to respond within a time limit. There are three states that the system can move into: a timer will move it into the Omission Timeout state, but before this occurs, a response from the rat can move the system into a Delivering Reward or a Incorrect Response Timeout state. (Maybe in this example the two timeouts do not need to be distinguished.)