Project 5: Chapter 14: Exercise 43

This exercise is based on the materials in the Sniffy Pro manual.  To get a feel for the software download the Sniffy Demo software available on this site, and print off this document to follow step by step.  Some users may find it helpful to run the Tutorial, and see Project 5 before they try it by themselves.

Shaping Behaviors Other Than Bar Pressing

Training Sniffy to perform tricks or other unusual behavior involves a number of steps, the same steps we used to train Sniffy to press bar in Project 2: Operant Conditioning.  This project will teach you how to train Sniffy to beg for food and introduce you to the Shaping Tutor, a teaching feature of the software that shows you how an expert would train sniffy to wipe his face, beg for food, or do a front roll.

Magazine Training: The first step is to establish a secondary reinforcer through a procedure resembling classical conditioning. The secondary reinforcer must be a stimulus whose timing you can precisely control. A common procedure used with dogs and many other kinds of animals is called ãclicker training.ä You get a little noisemaker and teach the animal that whenever it hears the clicker sound, you will give it a small treat.We will magazine train Sniffy to respond to the sound of the food being delivered, and that click noise will serve to reward Sniffy during training.

Shaping Through Successive Approximations: After the animal has been magazine trained, we use operant conditioning procedures to gradually, step by step, teach the animal to perform the target behavior.  Shaping is the technical name for a procedure used to train an animal to do something by reinforcing successive approximations of the desired target behavior. With real animals, the trainer leads the animal to progress through a long series of small steps. Reinforcement is delivered for progress and then withheld until more progress has been made. 

The exercises in this chapter will provide you with practice taking Sniffy through short shaping hierarchies (2 or 3 steps)  that culminate in behaviors that are fun to watch. Sniffy is easier to shape than a real rat, partly because he never becomes satiated for food and partly because his behavioral repertoire is smaller than a real ratâs. For each of Sniffyâs tricks (Target Behaviors) we have defined a shaping hierarchy÷a sequence of behaviors that progressively resemble the target behavior more and more closely.  The Target Behaviors that you can train Sniffy to perform are:

Bar Press	- pressing a lever on the back wall to deliver a food pellet
Beg	- standing and nodding facing the viewer with his paws up in front
Roll	- doing a front roll or somersault
Face Wipe	- grooming his face with three rapid wipes of his paw across his eye

Sniffy rarely, if ever, performs these tricks unless he has been trained on one or more of these rough approximation (or precursor) behaviors:

Rear Up Facing Back	- standing up facing the back wall with the food hopper
Rear Up Facing Front	- standing up facing the viewer
Head Lower	- lowering his head between his hind legs (i.e. licking his genitals)
Head Tuck	- tucking his head under and rolling back and forth
Face Touch	- grooming his face twice with his paw

For example Sniffy does not naturally do front rolls, but after magazine training we can start shaping by rewarding a similar move he does naturally (lowering his head between his legs) that approximates the Target Behavior (roll). When he has been reinforced for lowering his head sufficiently you will see the rocking tuck move emerge occasionally. When he starts to lower his head frequently, then the second successive approximation (rocking with his head tucked under) becomes the move that gets reinforced, until he does that a lot. Over time gradually stop rewarding him when he just lowers his head.

Finally, when Sniffy is tucking a lot he may suddenly fall forward and do his first front roll.  Then you can start rewarding tucking and rolling until he rolls frequently. To get him to roll a lot you have to stop rewarding tucking eventually. We call Head Lower a precursor of Head Tuck, and Head Tuck a precursor of  Roll not only because they are successive approximations of rolling but because Sniffy has to lower his head in order to tuck, and he must tuck back and forth in order to roll.

The Shaping Tutor: To ensure that you succeed in shaping Sniffy to perform these behaviors if you try, we have provided lots of unrealistic help. You can tell Sniffy Pro program to automatically record and reinforce target behaviors even though the behaviors do not involve interacting with a device in a way that would permit automatic recording with a live animal.  It is as if we have an expert viewer reporting every occurrence of the move to teach you how to shape Sniffy.

The Shaping Tutor can be set to record and reinforce any of the four Target Behaviors, and five precursor behaviors shown under Shaping Tutor.  To set the Shaping Tutor select the Design Operant Conditioning Experiment command from the Experiment menu, click on Continuous reinforcement, click on Continuous Reinforcement, and pull down the menu called Reinforcement Action.

Reinforcement Action menu: You can select which behavior you want to automatically reinforce under two headings:

Target Behavior:  No Reinforcement, Bar Press, Beg, Roll or Face Wipe

To train Sniffy to Face Wipe reinforce Sniffy (give him food) whenever he touches his face. It is advised to select Face Wipe from the Reinforcement Action menu so that every time, while Sniffy does the novel trick, the program will reinforce Sniffy with the click sound and drop a food pellet in the hopper.  Once he does it a lot you he will finish training himself.  It might take 30 to 45 minutes to get Sniffy to do the trick repeatedly, so save often so that you can take breaks and continue from the same spot.

Shaping Tutor: Rear Up Facing Back, Rear Up Facing Front, Head Lower, Head Tuck, Face Touch

If you are unable to reinforce Sniffy effectively by hand, select Face Touch from the menu and watch what is going on as the shaping tutor automatically reinforces the precursor move called Face Touch. When you believe that you are capable of effectively reinforcing these behaviors manually, reopen the Design Operant Conditioning Experiment dialogue box to reselect your Target Behavior, Face Wipe.

If you want to see how an expert would train Sniffy to Face Wipe, select Face Touch from the Shaping Tutor section of the Reinforcement Action menu.  If the rat is magazine trained he will soon start to touch frequently and you can select Face Wipe from the Target Behavior Section to complete training.  This auto training feature is much faster if you then Isolate Sniffy to accelerate time.

Measuring Behaviors and Recording Results

You might consider training complete when Sniffy has performed the target behavior four or five times in a minute, but this will vary for each behavior. You need to know a baseline of how often he performs a move before training and how often he does it after each stage of training.  We use the data from the cumulative record, the operant associations window and other screen in the Windows menu.

The Cumulative Record will tag and record whatever behavior you have specified in the Reinforcement Action menu (or nothing at all if you specify No Reinforcement).  The pen moves up every time it occurs. You can set the program to simply record and not reinforce the action by clicking on the extinction button.  This is useful for baseline measures of behavior before training.  You calculate response rates (times/minute) based on the 5-minute lines, and the pen resets every 75 counts.   A quick measure is 75 divided by the number of minutes between consecutive pen resets, or, count the number of ticks between two 5-minute lines and divide by 5.

Operant Associations mind window: Once Sniffy has performed the target behavior four or five times in a minute, you can stop shaping him and watch the progressive effect of reinforcement as Sniffy does the action more and more often. At that point, you should start keeping an eye on the Action Strength bar in the Operant Associations mind window to observe the development of his learning.  When the bars get over 50% they are just beginning to learn, over 75% is intermediate, and over 95% is expert.

The Sound-Food association must always be high (over 75%) to indicate that he is still magazine trained and associates that sound with food.

The BarÐSound association (only relevant when you are recording bar presses) tells us how well Sniffy knows that the bar is the device he uses to produce the sound that signals the delivery of a food pellet.

The Action Strength is the degree to which Sniffy has learned that performing the target behavior produces the sound that signals the delivery of a food pellet.

Disclaimer: The exercises in this chapter do not provide a realistic preview of what you can expect if you ever try to shape a live animal. There are no Mind Windows with real animals!

• To train Sniffy to roll or face-wipe, you begin by reinforcing grooming movements with a sound associated with food. Yet research with several kinds of mammals has shown that presenting food or secondary reinforcers associated with food is not an effective way of increasing the frequency of grooming movements (e.g., Shettleworth, 1975).
• Reinforcing a precursor behavior will cause Sniffy to start doing something else that is qualitatively different from (and we think more interesting than) the precursor behavior. In contrast, shaping a real animal ordinarily involves leading the animal through a large number of small quantitative changes that add up to a major change in behavior only after a protracted program of training.

Nevertheless, we believe that practicing with these simple examples will help you learn the principles of shaping. Just donât expect a real animal to be as ãcooperativeä as Sniffy.

Exercise 43: Shaping Sniffy to Beg

To see what the behavior that we call begging looks like, open the file named BegDemo in the Sample Files folder. Once youâve seen what the target behavior looks like, follow the steps below to train Sniffy to beg.

• Open the file from Exercise 22 that we suggested you call Ex22-MagTrain.
• Select the Save As command from the File menu to give the file an appropriate new name (such as Ex43-ShapeBeg) and save it in the Sniffy Files folder on your computerâs hard drive. Saving the file with a new name before you start shaping Sniffy preserves your original magazine-training file for future use. If your first attempt at shaping is unsuccessful, you can go back and try again without having to magazine-train Sniffy again.
• If Cumulative Record 1 is not visible, make it visible by selecting it from the Cumulative Record section of the Windows menu.
• Select the Design Operant Conditioning Experiment command from the Experiment menu.
• In the dialogue box that appears, select Beg from the Target Behavior section of the drop-down menu under Reinforcement Action. After making the selection, click on the OK button to close the dialogue box.
• Note that the terms CRF and Beg appear in the Cumulative Record window. This notation means that the program will automatically reinforce every instance of begging and that the cumulative record will record begging movements.
• As your first approximation to begging, give Sniffy a pellet of food whenever he rears up facing forward (that is, facing you) anywhere in the operant chamber.
• After you have reinforced rearing up facing forward a number of times, you will begin to see instances of begging; and the program will automatically reinforce these.
• If you are patient, the time will come when Sniffy will beg 10 or more times during a 5-minute interval. At that point, you can stop shaping, sit back, and watch the progressive effect of reinforcement as Sniffy continues to beg more and more frequently.
• Sniffy is fully trained when the Action Strength column in the Operant Associations mind window approaches its maximum with Beg as the automatically reinforced behavior. At that point, select the Save command from the File menu to preserve your trained Sniffy for future use.

After 45 to 60 minutes of attempting to shape Sniffy according to the instructions given above, Sniffy should be begging at least 10 times during each 5-minute interval delineated by the alternating solid and dotted vertical lines in the cumulative record. If your attempt at shaping fails to obtain that minimum result, something is wrong with what youâre doing.

One possibility is that Sniffy has not been properly magazine trained. Look at the Operant Associations mind window. The soundÐfood association should be at its maximum level on the scale. If it isnât, either go back and repeat Exercise 22 to create a properly magazine-trained Sniffy or use the file named MagTrain from the Sample Files folder. A second possibility is that you may not be reinforcing instances of rearing up toward the front wall as outlined above, or not reinforcing these behaviors quickly enough.

Using the Shaping Tutor

For users who encounter difficulty with shaping, we have provided a section of the Reinforcement Action menu in the Design Operant Experiment dialogue box called the Shaping Tutor. If you are having trouble reinforcing Sniffyâs rearing behaviors:

• Choose the Design Operant Conditioning Experiment command from the Experiment menu.
• Under the Shaping Tutor subsection of the Reinforcement Action menu, choose Rear Up Facing Front.
• The Sniffy Pro program will now automatically reinforce all instances of rearing up facing the front of the cage, including all begs. Watch which actions the program automatically reinforces.
• Once youâve figured out what you were doing wrong, reopen the Design Operant Conditioning Experiment dialogue box and choose Beg from the Target Behavior section of the Reinforcement Action menu.
• Alternatively, if you are still unable to reinforce Sniffy effectively by hand, watch what is going on as the program reinforces front-facing rears automatically. Once Sniffy begs 10 or more times in a 5-minute period, reopen the Design Operant Conditioning Experiment dialogue box and choose Beg from the Target Behavior section of the Reinforcement Action menu. The program will complete Sniffyâs training automatically.
• When Sniffy is fully trained, save the file.

Some Things to Do

To ãautotrainä Sniffy to Bar Press, select ÎRear Up Facing Backâ to reinforce all instances of rearing up facing the back wall, including all bar presses. Watch which actions the program automatically reinforces and the changes in the CR and Operant AssociationsWindow. Once Sniffy presses the bar 8 to 10 times in a 1- or 2-minute period, reopen the Design Operant Conditioning Experiment dialogue box and choose Bar Press.  When the Action Strength bar in the Operant Assocaitions window goes over 75% he has effectively learned the move. Complete to 95%.

Measure Sniffyâs Behavioral Repertoire

Roughly speaking, the items in the behavioral repertoire of an untrained Sniffy can be categorized into eight major components:

• Paw lifting: Sniffy lifts his front paws off the floor anywhere in the operant chamber. Do not include bar pressing in this category.
• Face touching: Sniffy touches his face with one of his front paws.
• Head lowering: Sniffy lowers his head down between his back legs as if he were licking his genitals.
• Sniffing: Sniffy stands with all four paws on the floor and twitches his whiskers. You can see his whiskers twitch when he is facing either toward you or toward the left or right wall of the operant chamber. When he is facing the back wall, you canât see his whiskers, but you can see him wiggling a bit.
• Locomotion: Sniffy walks around or turns his body.
• Bar pressing: Sniffy presses the bar.
• Hopper sniffing and eating: Sniffy sticks his head into the food hopper. If a food pellet is there, he eats it.
• Drinking: Sniffy places his mouth against the side or end of the drinking spout and makes licking movements (similar to sniffs).

Training Sniffy to press the bar modifies the frequencies with which he performs many of the behaviors in his behavior repertoire. To determine how the frequencies are changed, first watch a completely untrained Sniffy for 15 minutes and record how frequently he performs each kind of behavior. (To produce a completely untrained Sniffy, simply open a new file.) Then watch and record the behavior of the completely trained Sniffy that you produced in Exercise 23 for the same length of time.

• How much more often does the trained Sniffy press the bar, compared to the untrained Sniffy?
• Which categories of behavior occur less often in the trained Sniffy?
• Are the frequencies of any categories of behavior unchanged?
• Do any categories except bar pressing increase in frequency?

A major component of this exercise is figuring out how to set things up so that you can accurately and objectively record the frequencies with which Sniffy performs the various categories of behavior. It is a good idea to work with a partner, where one person watches Sniffy continuously and calls out the behaviors as they occur while the other person enters the data on paper.

Thank you for exploring Project 5, an advanced operant conditioning project in which you train Sniffy to perform one of three tricks.  Now teach Sniffy to lower his head, rock, and roll!