E420: 샤론 캐럴(Sharon Carroll) - "보상 줄이기에 대한 현실적인 시각"

giving a treat every time they perform the behavior um even if it's just to produce a reliable pet dog behavior we still really usually want to uh shift to that intermittent schedule because when a behavior is on a continuous schedule then when we miss a reward just even for several repeats of the behavior in succession uh we can see this rapid deterioration in the quality and reliability of that behavior but once we've put that behavior onto an intermittent schedule of reinforcement then we have options we can still reward on the really high rate of reinforcement but if we do miss reinforcing for several reps of uh that behavior in succession it won't result in any deterioration in the quality or reliability of the behavior um there absolutely will be behaviors though that need to be maintained on that very high rate of reinforcement um where we sort of reinforcing many of the performances um of the behavior and then other behaviors where the rate of reinforcement can actually be very low so only a very small percentage of the performances are reinforced uh or potentially even the reinforcement will be provided in a way other than from us so it may appear like the behavior doesn't need reinforcing at all because we don't need to keep handing over treats and toys but really it's just that it's being maintained through some other way um the reliability is being maintained some other way interesting it's really kind of an interesting way to think about it um i'm sure we'll circle back to that but how do you kind of decide for those behaviors that you are going to reduce reinforcement when the behavior is at that point like when do you look at something and go okay this is a solid enough behavior that it's time to start reducing reinforcement or maybe you start even before it's a solid behavior i don't know yeah well there's a scientific answer to that question uh where we're referring to what's actually happening in the brain and then there's the more practical answer where we're just assessing directly from the behaviors we're seeing in front of us so there's actually three phases that occur when our dog's learning a new trick or learning a new behavior there's the acquisition phase and that's when the brain doesn't really even have a clue what's happening the dog doesn't know what's what's going to end up in reinforcement they're just sort of guessing they're just throwing out random guesses and trying to work out what gets rewarded and what doesn't so they have no clue and the brain is madly trying to

identify some sort of association between oh when i did that behavior i got a reward and then over time we're going to reach that next phase where we get that action outcome phase where the associations are now being identified so the brain's going i'm pretty sure every time i do that behavior i get that reward but the brain's going to keep constantly monitoring at that point every time that behavior is performed they're going let me check am i getting that reward because that's what's telling the brain that that was the correct choice that was the correct behavior for that cue that is the correct behavior and that's what's going to get rewarded then we move through to this habit formation phase and that's where the brain actually goes okay the association is really clear now i do understand that when that cue happens i should do this behavior in order to get the reward and so this is really clear associations being formed and the brain's now able to monitor less closely so it's sort of like okay i know that i'm right now i don't need to monitor every time this behavior is performed um so it's more um the behavior is going to be performed just in response to the cue without as having without the dog having to consciously really make a decision and really think about what behavior to perform and um the brain's sort of pretty sure that that's the right choice so it's it's not going to keep monitoring now from a practice or not monitor as closely it still keeps monitoring but just not every rep and so from a practical perspective in the early phase we just see lots of guessing some correct some incorrect uh we're still doing the luring or the shaping we're using props that's certainly not a time to reduce reinforcement in that next phase we're going to see lots of accurate responses happening um but when we skip a reward there may still be an immediate change to our dog's behavior in that next rep whereas when we reach that end phase what actually happens now is the behavior is occurring rapidly and accurately in response to our cue we see that happening so without rewards being apparent before the behavior is performed but still giving a reward after every performance of the behavior but importantly when we can sit at that point we can start considering skipping a reward after one of the reps and if the next rep is unaffected then we can start the process of reducing reinforcement i like that way of thinking about it of actually being able to test okay does the behavior change as we reduce reinforcement and if not then we've kind of gotten it solid we're at

the right phase in the process exactly and that's remembering that it's always that one rep we skip one rep and we go back to rewarding and we skip one rep and go back to if we start skipping multiple reps we might see some disintegration anyway but just that one rep that's what gives us information we skip that one rep and we look at what happens that very next time we deliver the cue can you give us kind of an overview of how you actually approach reducing reinforcement so you've done your test you got that one rep and the behavior did not degrade what next okay so yeah we start at that point we check the behavior is established so we know the couple of things we want to check here is that we don't need treats or toys present before the behavior is performed now that's separate really to anything to do with reducing reinforcement but a lot of the time when we look at just problems that are occurring with people trying to get reliability from behaviors it's because there's those they haven't even reached that point yet so we don't we definitely don't start reducing reinforcement until the behavior happens reliably and accurately in response to the cue without us having treats and toys present before we cue that behavior so that's the first thing and so once we know that behavior is rapidly and accurately occurring on our first cue without the need for anything any treats or toys before it then we can skip that one reward now we analyze what happens okay great if the behavior just is solid nothing changes the very next rep everything's the same now we can start to shift to that intermittent schedule but we still keep that rate of reinforcement very high so we might be only missing one out of every 10 reps for a little while and then we can start skipping two but maybe not even two in a row too close together maybe do one with with a reward one without one with a reward one without and then we might gradually move down to where we're on the opposite sort of end of the spectrum where we can ask for nine or ten behaviors before we even need to give a reward but we do need to do that process gradually in that in that process though if we miss a rep and we see any delay in response to the cue or any change to the accuracy or enthusiasm we just sort of abort the mission temporarily and go back to rewarding every time interesting all right so how do you decide i know you kind of answered this there but you know if you're moving too quickly you're not quickly enough when you're moving you know the food

or the toys from the equation how do you kind of keep an eye on that or like how do you monitor that you're not moving too fast or not moving quickly enough yeah well if you're moving too quickly then we'll see signs of confusion and frustration we'll see and that frustration depending on the individual dog that frustration might result in quitting it might result in them just walking away going i don't want to do this anymore or it may result in them getting amped up they might be barking at us mouthing our clothing doing zoomies around the room so if we're seeing signs of confusion or frustration we've tried to reduce reinforcement too quickly either we've we've started before the learning process was complete or we're just trying to get that rate of reinforcement down too quickly like the actual rate of rewarding down too quickly and also we're going to see reduced reliability so maybe they need repeated cues to get the behavior sometimes they're responding accurately to the cues sometimes they're not or there might even be a lag time between the cue and the behavior so we give the cue there's a pause before they do the behavior all of those things would tell us that either we've started to try to reduce reinforcement before that behavior is established before it's reached habit formation phase or we've got onto that intermittent schedule but we've been in too much a rush to get down to a much lower rate of reinforcement instead of sort of doing that quite gradually now fortunately though there's never a point where we've waited too long before starting reducing reinforcement so a lot of people think it's like oh you know you could beat um you can start it too quickly but maybe you could also start it too late like you've stayed on that continuous schedule of reinforcement for so long and now your dog's reliant on it but that's not actually what happens it's not that we've waited too long it's that people then sudden do a sudden shift so no matter how long we've waited if we've kept that behavior on a continuous schedule for a long while then we need to incrementally still shift onto that intermittent schedule not try to make that shift really suddenly or try to drop that rate of reinforcement really quickly because that's when the problems actually occur um we still need it to be systematic and incremental um and i think at this point it's probably worth talking about like i keep saying continuous schedule an intermittent schedule continuous schedule of reinforcement just means one for one every time they do the behavior they get a treat

an intermittent schedule just meaning not every time they do the behavior so sometimes they do the behavior and they get a treat like the treat comes sometimes they do the behavior and they they don't get a treat that time and if we look at it from a human perspective um it's often talked of as that continuous schedule of reinforcement being like a vending machine like where we go up we perform the behavior of putting the money in and hitting the button and we expect a treat every time because that's what it's a continuous schedule same with a light switch we walk in the room we flick the light on we expect to see the reinforcement of light because that's what we want for our putting a light switch on continuous schedules are great for learning new behaviors because it's very easy for the brain to identify that association between the cue the behavior and the reward because the reward's happening every time but also extinction occurs really rapidly because the reward is expected so if we if we sort of miss too many in a row we will get extinction of that behavior the behavior will deteriorate and won't get such good quality behavior will lose some reliability um also the risk of frustration-based behaviors when rewards aren't delivered is really high when we're on a continuous schedule so you know how many times people start sort of banging on a vending machine because they want the treat because it was guaranteed it was expected and it should have come and so now we get these and it's the same with our dog if we if we're on that continuous schedule and we are not delivering the rewards and we're skipping more than just one that first time that can be hard um on them so we might see frustration-based behaviors now the intermittent schedule a lot of people in the human terms will refer to that like a slot machine we put the money and we hit the button and we know that sometimes we're going to get rewarded not every time we perform that behavior now intermittent schedules of reinforcement are really great for um producing some persistence some resistance to extinction so you will keep hitting that button multiple times in a row and just because it doesn't get rewarded every time it doesn't deteriorate your desire to keep pushing the button and that's what we want from our dog is we really want that behavior to just every time they hear that cue they perform that behavior really persistently really reliably because in their mind they're still sure at some point it is going to be reinforced and that's what we're trying to get um what we're trying to have happen when we switch from

that continuous schedule to the intermittent is just to help them build up that understanding that it will be still reinforced but it's just not going to be reinforced every time so just persist just keep going just perform the behavior and at some point that behavior will be reinforced and i think it's easy for us from a human perspective to look at those examples and not realize how complicated that is for our dogs because for humans we use language like verbal and written language to explain these things ahead of time someone told you that's how a vending machine works someone told you that's how a slot machine works but animals need to learn that through experience so if a behavior is on a continuous schedule there's an expectation of a reward we need to shift that behavior onto the intermittent schedule very gradually until they go oh okay so maybe just every five or six times it's going to be rewarded maybe every 10 or 12 and we can't suddenly go to 40 still because we're going to get a deterioration we have to build that gradually um so remember just remembering that habit formation really just means the behavior will happen in response to the cue without needing conscious assessment or decision making and the brain monitors less frequently but it still monitors so if we walk into the room into a room and we turn the light switch on and the light doesn't come on okay we work out that it's blown globe but when we go in that room again we're going to turn that light switch on again because it's a habit it's reached habit formation and we do that quite a while before we walk in the room and think before we turn the light switch on we get oh that's right no point the light's not working so it's important to remember extinction still happens it just happens more slowly once we've reached that habit formation phase um and whereas if for example someone gave you a button that wasn't guaranteed to have a light come on so it gives you a torch and you press it the first time torch doesn't work and then you press it a second time torch works and then next time you use it it's like you have to press it three times and then it's two times and then five times but when you know that that's how it works you will start just pushing that button multiple times um to get and you won't be as frustrated as you would be with a light switch or a vending machine that's not working anymore so i don't know if that helps people to understand that that the concept at all between continuous and intermittent but hopefully it helps someone to understand it yeah no i think that i

think that was a good explanation does any of this change depending on what behavior specifically we're looking to reduce reinforcement for uh fundamentally no the process is is the same but some behaviors may need to be trained for longer before we reach that end point so some behaviors are really simple within a couple of sessions that's they're already at habit formation the dog knows oh i do you do the cue i do the behavior i get the reward okay we're ready to start reducing reinforcement but whereas other behaviors or different types of dogs we might be working for weeks and weeks sometimes months before we're really ready to start reducing reinforcement for that behavior um so and and different behaviors will require different rates of reinforcement to maintain them as well so a very easy behavior won't need the same rate of reinforcement as very complex behavior and it's also going to be affected by things such as the innate traits of the individual dog so a high energy dog doing a really fun thing is going to need a very different rate of reinforcement to a very low energy dog being asked to do a high effort high physical effort they're going to want that reinforcement fairly frequently to maintain it and you know same as if there's an intrinsic value in the activity that dog really just loves doing that thing you're not going to need to probably give them treats or toys once they've learned the behavior or you may certainly only need to do that very infrequently um same with a dog that might love the challenge and complexity of work versus a dog that doesn't love that so much so our rates of reinforcement will vary with things like that um also the familiarity of the behavior a very new behavior might still be on a very high rate of reinforcement whereas a very familiar established behavior might be way down on a very low rate of reinforcement um and even the influence of the environment maybe um we increase the rate of reinforcement for a specific behavior just to help increase the reliability of that behavior because the environment's more challenging for that dog at that time it's kind of an interesting thing to think about from the standpoint of a lot of the times we are seeking to reduce reinforcements that we can have those behaviors hold up in the more challenging situations yeah it's like yeah but but we have to we have to separate that in pieces it's like okay initially in that challenging environment we increase the rate of reinforcement then eventually we do decrease back to our lower rate of reinforcement but yeah it is common people go out into those bigger

environments and they also want to stop using the treats and toys at the same time yeah and you can see why that maybe makes it really hard to accomplish your own goals which is you know where i was going to ask you next which is why do you think so many teams kind of struggle with removing reinforcement sounds like maybe that's part of the answer yeah yeah i think um sometimes thinking about their end goals too much uh before they even start training the behavior they're already worried about like right well i need to really not have this need too much reinforcement you're like no no the dog's learning the behavior at this point they need lots of reinforcement and so that like trying to limit the rewards too soon in the process or not building motivation to perform the behavior enough um not building enough a lengthy enough history of reinforcement before they're quickly trying to drop out the treats and toys um yeah uh maybe they they still have rewards present before cuing the behavior that interferes a lot with the situation so we need to have stopped that happening before we even reduce reinforcement um so reducing rate of reinforcement too quickly is a big reason why people have problems suddenly skipping too many rewards in succession and like you said that's common in trialing they'll they'll keep the rewards up in training then they go to a competition and suddenly drop all the reinforcement in one go um so and i think the other thing is confusing working for longer periods of time with reducing reinforcement they're two very different things reducing reinforcement is specific to reinforcing individual behaviors every behavior is on its own intermittent schedule of reinforcement working for longer periods is about mental stamina that's about how long can a dog work before they reach what i refer to as the ctp the critical time point the point where there's going to be deterioration there's going to start to be some distractibility they're going to start looking for something better to do they're going to look around start sniffing the ground you know stop performing the cue behaviors but that's not to do with the rate of reinforcement or anything reducing reinforcement that's to do with building that mental stamina that ability to work for longer uh between rewards which is actually yeah a little bit of a separate a separate thing often it does get tangled into reducing reinforcement but it is actually quite separate um and i think yeah just not preparing for long periods of work

without treats and toys before entering the competition ring so people just you know sort of think their dog won't notice that they're just suddenly working for you know three minutes without any treats and toys it's like yeah if you can't do it in practice reliably frequently then you know probably not the best thing to just try and do it for the first time in the competition ring um and i think also i guess somewhat related issue um is not training an end of exercise cue so a lot of people have an end of session cue like you know all done do a treats gutter but not training an end of session end of exercise cue can be really tricky um when you're in a sport that requires you to reset and do multiple different exercises because a lot of people end every set every section of work with a reward marker so they work for maybe longer and longer and longer periods but then they end with throwing a ball or end with a strike on a tug or end with giving treats end with a treat toss it's like but what are you going to do when you're in the ring and you just need to end that exercise and then walk off and set up for the new exercise dog's gonna be going where's my reinforcement where's my treats where's my reward so training that end of exercise cue where the dog understands oh end of exercise okay i can have a little mental break we're going to go and reset for another exercise stops us from having to actually just work through the whole thing like i'm going to end this exercise then i'm going to heal to the next exercise you know we want that dog to have that little mental break and know that they're on a little bit of semi free time they can just move with us to the next exercise but they need to be able to do that without feeling like they they missed out on a reward um so yeah i think that's another important point where reason why people have trouble uh in competition rings when they haven't got a solid end of exercise um cue that doesn't involve giving treats and toys that's so interesting i mean we've pulled apart so many different pieces here right there's the reducing reinforcement for the individual behavior there's the building the mental stamina so you can do a whole routine of multiple behaviors that you have already reduced reinforcement for and then there's the piece of like okay yes you you're ultimately most competition venues or sports are going to require finishing one thing break next thing and again you have to transition all of those pieces without um classical reinforcers right so it's just so much more

complex i think than people who are new to dog sports take the time to really understand um so part of the reason we're talking about all this uh is you've got a new class right on reducing reinforcement on the calendar for august it is now open for registration people can go sign up um you want to share a little more about the class kind of which pieces there you're going to cover in the class and maybe who should consider joining you sure um so it's in the foundation foundation section because the content's not specific to one sport it's called progressing your training reducing reinforcement without reducing enthusiasm so even though reducing reinforcement is in the title the class covers like a lot more than just reducing reinforcement so in week one we do a lot of the building of the tricks and the behaviors so we're talking about shaping and luring uh we get the behaviors happening on cue um we're assessing current tricks and sort of saying okay which category do they fall into are they uh still at the point where you know they're they're at that acquisition phase really or the early action outcome phase so we're still doing lots of luring and um shaping they're not ready to put a cue there are they at the next level where we're kind of ready to put a formal cue in there um are they at the point where they actually are reliably happening but only when the treats and toys are present before the tricks queued and so we look at that as a separate uh issue that we work with um we look at is everything going well but we're still using a lot of props so we're talking about um that as being a potential um thing we need to work on we actually work on that in week two but we're assessing those uh exercises in week one we look at uh are we at the point where maybe it's that the trick's really really good but when we go into a different environment suddenly it all falls apart okay that's something we deal with later in the class as well and then we're also saying hang on there's a problem with this trick so the gold students will be saying i've got this trick and i don't know why it's not working but something's going wrong when i don't have the treats and toys the trick's not working and so we're going to be um looking you know maybe it's not always um accurate it's not always occurring on the formal cue or maybe it sometimes needs repeated cues so we're going to look at those sorts of things and that's in our week one week two is all about reducing reinforcement and using and fading props week three is about creating and maintaining

motivation and rehearsing like reward events and effective use of reward placement week four is about using remotely placed rewards so having our dog being able to leave rewards behind in an unconflicted way and go out and work and then return to them and we're talking then about building mental stamina working for longer periods using all these behaviors that are already on an intermittent schedule but we're piecing them all together and being able to work for longer and longer periods um and we're talking then about delayed rewards for that reason as well week five we're talking about um the training the end of exercise cue that we talked about and eliminating anticipation of cues so training and reinforcing the behavior of waiting for a cue which is a big one for a lot of dogs especially when we're using props we just put the prop down they just start doing the behavior and then we're like hang on we want them to wait until we cue it so we talk about that uh and we talk about preparing for competing without treats and toys and then in week six we just talk about working around treats and toys so when we want to be able to keep those behaviors reliable and occurring in a focused way even when there are treat containers nearby or toys on the ground or interesting smells or drop food so that's what we're talking about in uh week six so yeah it's um pretty jam-packed it's a pretty packed class yeah and i think you did also ask who might it suit um i think it's gonna suit i honestly think i mean everyone i'm sure everyone you talk to would go well my class will suit everyone and but honestly i think this class will be great for a big range of people so people who are brand new to dog training or brand new to positive reinforcement based training as well as experienced trainers looking to further refine their skills um i think it's great for people who instruct probably dog training classes as well or sport dog classes because the theory topics provide a lot of solid understanding of why we might make different choices with different dogs um and i think the theory uh for the experienced people will let them get their geek on because there's lots of really cool you know i think i'm just grabbing for some pieces of paper here but like week one we talk about cue salience and overshadowing and cue transfers and extinction and superstitious behaviors and all these sorts of fun things so uh i think the theory part for the experienced trainers will be really cool uh but there's also just uh the practical stuff is step-by-step

processes for the newer trainers and in fact i think in week one i even say if you're brand new to training maybe skip reading these you know certain three topics that are in week one um because you know i don't want you to get overwhelmed there's tons of other theory and if you're really keen you can go back and reread those um but the rest of it is very much practical step-by-step processes for um yeah people that are newer to training or people that are having current issues with something to do with their tricks that aren't working well all right final thoughts key points you want to leave folks with what else do we need to know on this um i think i just hop back to that it's never too late to reduce reinforcement i think a lot of people go oh i don't want to do it too soon but i don't want to do it too late and it's like well it's never too late like rewarding for too long is way better than trying to reduce the rewards too early or too quickly so the key is gradual shifting though from that continuous schedule to the intermittent schedule uh and treating each behavior in isolation so some behaviors will need to be maintained on that much higher rate of reinforcement than other behaviors so just yeah treating each behavior separately and just remembering it's yeah never too late to even if you've been just giving lots of treats all the time for everything and you're like oh no i've got in this horrible habit if i give treats all the time for everything um then come and join the class because it is never too late to change that uh to the situation where those behaviors are performed really reliably really rapidly and accurately without needing uh treats and toys after every performance of the behavior awesome all right well thank you so much for coming on the podcast to talk about all this no worries it's been fantastic talking to you again and yeah i'm looking forward to the class starting absolutely and thanks to all of our listeners for tuning in we'll be back next week don't miss it if you haven't already subscribe to our podcast in itunes or the podcast app of your choice to have our next episode automatically downloaded to your phone as soon as it becomes available today's show is brought to you by the fenzy dog sports academy special thanks to denise fenzy for supporting this podcast music provided royalty free by bensound.com the track featured here is called body audio editing provided by chris lang thanks again for tuning in and happy training mom