My Puppy, My Self

Your article on "The Negative Effects of Positive Reinforcement" assumes incorrect definition of terms.

Reinforcement is defined as anything that increases behavior. Punishment is defined as anything that decreases behavior. Positive is defined as adding something. Negative is defined as taking something away.

Positive Reinforcement (adding something that increases behavior) does not just mean giving food rewards. Indeed, food rewards can become punishing if the animal's appetite has been satiated. For a dog, a click can result in a game of tug, the door opening to be let out, and other consequences that reinforce the behavior preceding the click. Whatever the trainer identifies as a reinforcer (increasing the behavior preceding the click).

That said, the concept that positive reinforcement has an alternate side is correct. If the animal expects reinforcement but does not receive it, that can result in negative punishment (loss of reward, from the animal's perspective) which can be stressful.

To minimize stress, the training process should involve only very small shifts in criteria. The animal should be set up for 80 to 90% success rate (8 or 9 correct responses out of 10 opportunities for reinforcement). Many people miss this point, so what they consider "positive training" becomes laced with aversives. Animals do learn from aversives, but the emotional association of aversives can undermine the quality of the behavior.

What is a reinforcer is decided by the animal, not the trainer. The animal's internal and external environment, at any given moment, determine what is reinforcing. These environments are affected by genetics, learning history and precursor events. Is the animal thirsty? Is it frightened? Is it an adolescent or adult in breeding season? All of this is based on internal chemistry and external events. And all of it feeds into instrumental/operant behavior.

If the animal wants space (as many exotic animals do) then the trainer backing off is reinforcing. Aggressive behaviors are often reinforced inadvertently in this way (animal growls and bares teeth, trainer automatically steps back). This is not a food reward, but it does increase aggressive-display behavior.

Understanding the scientific meaning of the terms, rather than just their common usage, gives a better understanding of the science of operant conditioning.

Hi, Chris,

Thanks for your comment.

You're absolutely right about your definitions. What I was referring to, in a generalized way, is the way the +R training movement uses (or misuses) the terms they supposedly live by. You can't deny that in Karen Pryor's example, she's applying positive reinforcement, albeit using a variable rate, and that this can cause an emotional collapse. So in that sense, my article should have been titled, "The Negative Effects of Operant Conditioning," or "The Hidden Dangers of Operant Conditioning," or something along those lines.

I also stand by what I said, that while using survival needs may seem to reinforce behaviors, there's almost always a negative experience attached, even though the behaviors are being learned and repeated. It's as if the liver snaps taste great initially, but they also leave a bad taste in the dog's mouth that he doesn't notice until later. There is something in the dog's psyche that makes him want to respond to external rewards and markers just so that he can use his natural energy in a social context. But no matter how careful you are with the clinically-derived application of reinforcers, the dog still feels that something is missing.

That's why there are times when a dog's behavior will seem to have been successfully positively reinforced -- he's consistently obedient to a whole set of commands, all reinforced with food, clicks, and praise -- and yet the effects of his conditioning break down easily and completely when his prey drive is triggered. Thus, "learning in flow"(which, in your model, would probably be described as self-reinforcing) is a much better, and more successful approach to training dogs and to teaching kids.

Plus you can't teach police dogs -- attack dogs, detection dogs, search-and-rescue dogs -- successfully using only positive reinforcement. They won't work long hours for clicks and liver treats. They'll only do it when their prey drive is part of the learning process.

LCK

Have I misunderstood you? I have always understood that there is a clear distinction between the classic conditioning of Pavlov's dogs, where the operator is in control of action and consequence, and operant conditioning, where the dog/animal is in control of the action that then delivers a consequence, usually a reward. I accept that operant conditioning may under some circumstances be stressful for the dog - I know any number of humans who much prefer to blame the ubiquitous "them" than accept that they have any control over consequences of their actions - but I believe it is still an important distinction, not least when it comes to using classical conditioning for overcoming behavioural problems.

I believe dogs have different drivers preponderating - whether affiliative, prey, food or whatever. Food is probably the one thing that the majority of dogs respond too, and is therefore inevitably the reward suggested by +R trainers (although every book I've read or class I've attended has emphasised the importance of understanding what motivates your dog). For a dog with a strong prey drive - which is most of the breeds chosen for the tasks you describe - a reward that combines prey, play and affiliation is going to be a sure fire winner. For those of us with papillons and shitzus, the balance may be slightly different. Surely the emphasis should be on getting to understand what my dog finds rewarding, rather than forcing her to respond to different one-size-fits-all approaches?

Hi, Frances,

I agree that, on a surface reading of behavior, different breeds are motivated more by some things than others. However, the point I'm making is that no matter what external reward you use, the thing that makes one thing more rewarding than another, irrespective of breed type, past conditioning history, etc., is whether the external object, event, or marker addresses a very simple internal need: the reduction of the animal's internal tension or stress. This is why conditioning often breaks down when a dog is confronted with any object or event -- from seeing a squirrel, to hearing a distant fire engine siren -- that resonates with the dog on a deeply emotional level.

Anyway, that's how I see it.

LCK

As someone who includes the +R aspect of Operant Conditioning in my training of horses & dogs, I am really sorry to hear you've not been able to witness +R used properly. Like any training method, if it's applied inconsistently or improperly, of course the result will be sloppy behaviors and signs of frustration.

Don't forget some of the top agility, freestyle, and guide dogs are all +R trained -- and they can love their jobs. And yes, there are even SAR and police dogs switching over to the +R approach! The delay in the switch over is not because +R doesn't work; it's because the last generation of trainers of police dogs learned to train using other methods, so it takes awhile for a new idea to work its way into a tradition-based system.

I see you're concerned about variable reward schedules. But not all +R trainers use variable reward schedules. The ones I know tend not to use them. You name Karen Pryor as one of the leaders in the +R field. Yet when I attend her annual Clicker Expo, most speakers stick with is to avoid a variable reward schedule. I personally don't care for them. You cannot paint all +R trainers with the same brush. Heck, there are even "positive" trainers who don't click every reward - and some who refuse to use any bridge signal at all when rewarding.

Pavlov's dogs keep being cited, but they are an example of Classical conditioning: the dog salivates in the expectation of food when hearing the preceding stimulus (bell). Operant conditioning using +R is not the same thing. The +R trainer is using a cue first, then dog's behavior -> click -> reward. Or you can think of it this way: one is an unconscious association and the other is a learned behavior. I just wanted to make it clear they're not the same thing.

There is the description of a person who trains a dog to hump a leg for rewards. While humping may nor may not be a sign of happiness in the dog, having one person reinforce it, resulting in more humping, is at worst is a poor choice of behaviors to teach. It does not invalidate an entire training approach if you find one behavior that one trainer taught to be distasteful. I can use any method to teach "frustration" behaviors such as nervous circling or nuisance barking or shivering. We cannot glance at a behavior and assume we know why the dog is doing it, if we don't know the context.

The idea that dogs work for emotional rewards I would agree is true. But what the article doesn't make clear is that +R is not strictly food. Rewards are anything the animal finds desirable: rubbing, playing, attention, praise, a toy, smelling a pleasurable scent, you name it. You can use any of them as the reward!

And while it's easy to find fault with another training approach, one should only do so when offering a better option. If you're throwing out Operant Conditioning, that means also throwing out punishment based methods (such as the currently popular Cesar Milan). If you're throwing out Classical Conditioning, that means there is no explanation why dogs show happy-excitement behaviors when I pick up their leash to take them out. What's left? The "Pushing" exercise you shared in your comments was interesting, but isn't it also just using aspects of classical and operant conditioning? Operant: dog is cued (eg. human stands a certain way/smell of food) , dog offers behavior (touch chest to human hand), reward is offered (dinner). Classical: dog begins to associate touch with something he really enjoys, helping a dog who fears touch.

Kevin Behan, author of the Pushing exercise, says it's some sort of magical "magnetic" "electrical" energy (but which cannot be measured or scientifically proven exists) which causes the dog's behavior. Why not believe it's supernatural? Or our own telepathic powers controlling the dog? Or reincarnated relatives in dog form trying to please us? Why don't we believe it - because it's science fiction, not science. What keeps pulling me back to the Operant Conditioning paradigm is that it is logical and scientific, even for trainers who prefer punishment or release-of-pressure over rewards.

Is there energy involved? Yes, there is in that the dog is excited to engage in the process. It becomes a "game" to the dog because it is so much fun for them to figure out how to get the next reward. The only danger of reward based training is you'll end up with a dog who WANTS to work and who tries to engage you in training sessions when you're trying to do other things.

For further reading:
http://www.clickertraining.com/
http://www.equineclickertraining.com/

Or see it in action at Clicker Expo
http://www.clickertraining.com/clickerexpo/

Hi Calico,

Thanks for your comments.

I've already covered most of your points, either in a number of other blog articles or in replies to previous posters. But I'll be happy to go over a few things again.

The reason more tracking dogs, search-and-rescue dogs, and police dogs aren't trained via food and clickers is because training them via rewards makes them unreliable. There have been a number of situations where dogs trained this way gave false positives or weren't able to find a body (Chandra Levy) or a missing person (Elizabeth Smart) or "found" explosives that turned out not to be there.

Dogs trained specifically via their prey drive don't make these kinds of mistakes. You may say the difference is more about whether a trainer is using operant or classical conditioning, but I would argue that it goes deeper than that. It goes to the heart of what makes dogs different from all other animals (except humans and members of the dolphin family).

There is also a major flaw in how Pavlovian conditioning is explained by science, which Kevin Behan makes clear in his new book. The general idea held by most people is that the dogs salivated because they made an association between the ringing of the bell and being fed. Behan points out that we don't usually salivate to the point Pavlov's dogs did unless we're actually in the act of eating. So the bell triggered a physical memory of eating, which resulted in the autonomic nervous system response of salivating. There was no association made between the two events, or expectation of food. The two stimulit were part of the same physical memory template that was created during the conditioning process.

Don't get me wrong, Pavlovian conditioning is a real process. But the explanation for why it works is off.

When it comes to understanding how learning takes place -- in all animals, not just dogs -- operant conditioning, particularly as used by dog trainers, or anyone else outside of a laboratory setting, is only a first approximation at best. (This means it's only a second or third approximation in the labratory.)

Did you know that most behavioral scientists don't concern themselves with the so-called Four Pillars of operant conditioning? Or that there is no way to determine, even in the most controlled environment, whether a behavior is learned through positive or negative reinforcement? (I go into this briefly in another article, "Why Dogs Pull on the Leash: Canine Cathexis + Chemistry + The Joy of Sex.") Here's an excerpt:

"Positive reinforcements are not actual, physical objects any more than my hypothetical drive to connect is. They're more akin to a function of statistics, measured solely in terms of a behavior's response strength. We can only know if a tangible object, such as a toy or liver treat, might or might not have provided the mechanism for reinforcement by interpreting the resultant behavior after the fact, through a +R lens. Since it's also possible to interpret any behavior through the opposite, -R lens[7], behavioral science loses credibility in this regard. Then, once you add the necessity for determining what kind of reinforcement schedule was at play (and there are far too many to list here)[8], it simply boggles the mind how anyone can say they know with any certainty, other than as a pure leap of faith, that any behavior of any kind has been reinforced, or what the mechanism of reinforcement actually was.[9]"

My commentary about Jean Donaldson training her dog to hump her leg wasn't about the flaws on the underlying philosophy of Skinner, it was meant to show that people who consider themselves "positive trainers" aren't always as positive as they think they are or advertise themselves to be.

Your comments on Kevin Behan's positions are, I would have to say, are a gross mischaracterization. There is nothing "magical" about energy. Every living thing on earth is, above all else, an energy system. You yourself have described shifts in a dog's energy when he's learning via operant conditioning.

Meanwhile, you've completely misunderstood the nature of "The Pushing Exercise," and how it shows, quite clearly, that operant conditioning is not a real phenomenon, but just an approximation of how learning takes place.

I suggest you read my article specifically about this exercise: http://www.psychologytoday.com/blog/my-puppy-my-self/201001/mice-and-mutts-iv-conclusion-all-dogs-are-good-dogs-heart

Its effects cannot be explained via the principles of operant conditioning. To summarize, by pushing for food, a dog's behavioral problems, of whatever stripe, start to gradually disappear. Just by pushing for food. Try to explain that via operant conditioning. (Or read the comments section of that article to see an ostensible academic behavioral scientist try and fail repeatedly.)

I would also recommend some of the previous articles in the "Of Mice and Mutts" series. Plus, the articles I've written in response to some of Ian Dunbar's recent blog comments about how complicated behavioral science techniques are ("The first gift that we can give to all animal owners, parents and teachers is to simplify the ridiculously ambiguous and unnecessarily complicated and confusing [behavioral science] terminology. Second, let's simplify the underlying theory by going back to Thorndike's original premise -- that behavior is influenced by [its] consequences.").

http://www.psychologytoday.com/blog/my-puppy-my-self/201009/toward-unified-dog-theory-understanding-pattern-recognition

http://www.psychologytoday.com/blog/my-puppy-my-self/201009/the-unified-dog-theory-ii-using-pattern-recognition-in-training

In that series of articles, I point out why animals don't learn through cause and effect, and provide scientific evidence that there's no causal relation between a behavior and a subsequent reward, but that learning is about pattern recognition, specifically recognizing changes in environmental patterns. (In effect, dopamine doesn't get released into an animal's bloodstream when said animal is getting or even "anticipating" a reward, but it does get released when that animal recognizes a new pattern, positive or negative.)

In a future article I plan to go into how this works in dog training, and provide several clear examples of behaviors that are learned solely through pattern recognition, with no operant conditioning involved.

Thanks again for your comments,

LCK

Footnotes for the paragraph from "Why Dogs Pull..." quoted above:

7) "Some stimulus changes associated with an increase in behavior are difficult to classify as [positive versus negative reinforcement], and the use of either description may be nothing more than an arbitrary and incomplete abbreviation for the ‘pre-change' and ‘post-change' stimulus conditions as well as for what transpires in between. For example, is a change in temperature more accurately characterized as the presentation of cold (heat) or the removal of heat (cold)?" ("Negative Reinforcement in Applied Behavior Analysis: an Emerging Technology," Brian A. Iwata, University of Florida, Journal of Applied Behavior Analysis, Winter, 1987.)

8) Here are just a few reinforcement schedules used by behavioral scientists: fixed ratio, continuous ratio, fixed interval, variable interval, variable ratio, differential reinforcement of incompatible behavior, differential reinforcement of other behavior, differential reinforcement of low response rate, differential reinforcement of high rate, etc. And these are all considered "simple" schedules!

9) With all that said, I would actually agree that when a dog pulls, his behavior is being reinforced, but that the reinforcement comes from the pleasure the dog feels when he's cathecting his pent-up energy onto objects of attraction. This explanation is actually much simpler, far less abstract, and doesn't rely on statistics or arcane mental manipulanda. Plus the solution is much simpler too: provide the dog with a stronger feeling of pleasure (i.e., a stronger cathexis) by playing with him, on his level, and the pulling behavior will begin to diminish in strength, and may eventually stop on its own. Meanwhile it's hard for a dog to form a cathexis with clicker, or with someone who's dominating him.