Implementing schedules of reinforcement

The Whisker SDK provides a simple tool for implementing commonly used schedules of reinforcement between triggering events (such as lever-press responses) and outcomes (such as reinforcement).

For simplicity, the events that trigger a schedule will be referred to as responses for the time being (even though they don't have to be; see Higher-order schedules, below.)

Fixed Ratio (FR) N: Every Nth response causes an outcome to occur immediately.

Fixed Interval (FI) T: The first response to occur after an interval T from the last outcome causes the outcome to occur immediately.

Variable Ratio (VR) N: Each response has a probablility of (1/N) of causing an outcome to occur immediately. Thus, on average the outcome occurs every N responses. (Note: there are several ways of implementing VR schedules, of which this is one.)

Variable Interval (VI) NT: The first response to occur after a variable interval of mean length NT causes an outcome to occur immediately. (This is implemented as follows: during the interval, at the end of every sub-interval of duration T, there is a probablility of 1/N that the interval will end and the outcome will be made delivered when the next response occurs.)

Variable Time (VT) NT: The outcome will occur immediately after a variable interval of mean length NT. (This is implemented as follows: during the interval, at the end of every sub-interval of duration T, there is a probablility of 1/N that the interval will end and the outcome will be delivered immediately. There is no response contingency in this schedule; it is noncontingent.)

Note: there is no 'fixed time (FT)' schedule – this is provided by the TimerSetEvent function.

Implementing schedules

The SDK implements schedules by producing an Event message (i.e. calling the _Event subroutine) when the outcome is to be delivered. This Event message will appear exactly like a message from the server (and will include a time stamp in ms). The triggering response (for ratio or interval schedules) will also appear as an Event message.

Note: Outcome event messages will be received by the _Event procedure immediately before the event that triggers them, and will contain identical time stamp information.

Schedules are added with the AddSchedule command, and removed with the RemoveSchedule command.

Adding schedules

       Whisker.AddSchedule <Type> <parameter> <triggerevent> <outcome>

The parameter will be N for FR,VR,VI and VT schedules. For FI schedules it is T (in ms). For VI and VT schedules the T parameter is set (in ms) via the SDK's .ScheduleTimebase property; for example:

       Whisker.ScheduleTimebase = 1000

       Whisker.AddSchedule wsVariableInterval, 30, "a", "b"

means that "a" will cause "b" on a VI-30-second schedule (with a 1000-ms granularity).

The VBRatio client, provided as a demonstration client, shows how a simple FR schedule can be implemented using the SDK.

Removing schedules

Whisker.RemoveSchedule <Type> <triggerevent> <outcome>

This command will remove any current schedule that matches the supplied criteria. A parameter wsAnySchedule is provided to match any schedule, and empty strings should be used to match any trigger event or outcome.

Thus

Whisker.RemoveSchedule wsAnySchedule, "a", ""

will stop any events scheduled to be caused by "a".

Whisker.RemoveSchedule wsFixedInterval, "", ""

will stop all FI schedules.

Removing schedules is not based upon messages to the server, and therefore has immediate effect, like KillEvent().

Higher order schedules

'Outcome' or 'consequence' events resulting from simple schedules can in turn be scheduled by a second call to AddSchedule, providing a simple mechanism to generate higher-order schedules. For example,

Whisker.AddSchedule wsFixedInterval, 30000,  "LeverPress", "CS"

Whisker.AddSchedule wsVariableRatio, 4, "CS", "Reinforcer"

Following this, LeverPress events will cause CS events on a FI-30-s schedule, and CS events will cause reinforcer events on a VR-4 schedule. A second order design can thus be implemented by responding correctly to "CS" and "Reinforcer" events in the _Event() routine.

Note: If you schedule an event to cause itself (either directly or indirectly, through a higher order schedule) on a FR-1 or similar schedule, your client will "hang" as it constantly sends itself messages. You have been warned....