Reading A

I hate eggs. It’s not just the taste of eggs I can’t stand. The smell, the feel, the very sight of eggs is enough to make me sick. Watching other people eat eggs can make me nauseous. It’s not that I’m allergic to eggs. I like all kinds of baked goods that are made with eggs. I’m fine with eggs as long as they are cooked into other foods so they are no longer recognizable as, well, eggs. But eggs themselves, especially runny eggs, fill me with disgust.

I wasn’t born with a disgust for eggs. Nor did I always dislike eggs. My parents tell me I was actually quite fond of eggs as a young child. But somewhere along the line I acquired an aversion to eggs; chances are I had an unpleasant experience with them. No, I don’t think I was chased around a barn by clutch of crazed chickens. Most likely, I had an experience in which eating eggs sickened me. Or perhaps I was forced to eat eggs when I wasn’t feeling well. In any event, I have no memory of it. All I know is that I hate eggs and have hated them for as long as I can recall.

I described my aversion to eggs to introduce you to the topic of learning. Some responses, such as pulling your hand away form a hot stove, are reflexive. We don’t learn reflexive responses; we are biologically equipped to perform them automatically. Other behaviors develop naturally as the result of maturation. As a child’s muscles mature, the child becomes capable of lifting heavier weights or throwing a ball a longer distance. But other response, such as my aversion to eggs, are acquired through experience.

Psychologists take a broad view of the concept of learning. To psychologists, learning is more than just book learning or learning that takes place in a classroom. Psychologists generally define learning as a relatively permanent change in behavior that results from experience. It is through experience that we learn about the world and develop new skills, such as riding a bicycle or cooking a soufflé. Acquired taste preferences or aversions, including my aversion to eggs, are also learned behaviors. Note the use of the term relatively permanent in the definition of learning. For learning to occur, changes in behavior must be enduring. But change need not be permanent. It is possible to unlearn behavior. For example, you would need to unlearn the behavior of driving on the right side of the road if you wanted to drive in a country where people drive on the left side of the road.

Learning is adaptive – it enables organisms to adapt their behavior to the demands they face in the environment. Through learning, organisms acquire behaviors that increase their chances of survival. Even taste aversions can be adaptive. They prevent animals, including humans, from eating foods that have sickened or poisoned them in the past. But not all learned responses are adaptive. My own aversion to eggs limits the range of foods I might enjoy. By and large, however, learning helps prepare organisms to meet the demands that their environments impose on them.

Psychologists study many forms of learning, including three major types that are the focus of this chapter: classical conditioning, operant conditioning, and cognitive learning.

• What is learning?
• What is classical conditioning?
• What roles do extinction and spontaneous recovery play on classical conditioning?
• What roles do stimulus generalization and discrimination play in classical conditioning?
• What stimulus characteristics strengthen conditioned responses?
• What is a cognitive perspective on classical conditioning?
• What are some examples of classical conditioning in humans?

Do your muscles tighten at the sound of a dentist’s drill? Do you suddenly begin to salivate when passing by your favorite bakery? You weren’t born with these responses – you learned them. But how does learning occur?

To understand how responses are learned, we need to consider the work of the Russian physiologist Ivan Pavlov (1849-1936). Pavlov discovered the form of learning we call classical conditioning. Pavlov, who at the time was studying digestive processes in dogs, made this discovery when he observed that dogs would salivate to sounds in his laboratory that had become associated with food, such as the sound of metal food carts being wheeled into his laboratory.

You can think of classical conditioning as learning by association. If you associate the sound of a dentist’s drill with pain because of past dental treatment, the sound of the drill will probably cause you to respond with the muscle tension that is a natural reflex to pain. If you associate a certain bakery with a particularly tasty treat, you may find yourself salivating simply by driving by the bakery. In other words, you learn to connect or associate two stimuli – the sound of the dental drill and pain, for instance (Chance, 2009). Although classical conditioning is a relatively simple form of learning, it plays important roles in our lives - as you will see in this module.

Principles of Classical Conditioning

Pavlov performed many experiments in classical conditioning. In a typical experiment, he harnessed dogs in an apparatus similar to the one shown in Figure 5.1. When food is placed on a dog’s tongue, the dog naturally salivates. This reflexive behavior is called an unconditioned response (UR or UCR) (unconditioned means “unlearned”). A stimulus that elicits an unconditioned response – in this case, the dog’s food – is called an unconditioned stimulus (US or UCS).

Figure 5.2 outlines the steps involved in a Pavlovian experiment. As you can see in Figure 5.2b, the introduction of a neutral stimulus (NS), such as the tone produced by striking a tuning fork or ringing a bell, does not initially elicit a response of salivation. It may produce other responses, however. A dog’s ears may turn up in response to the sound, but the dog doesn’t naturally salivate when it hears the sound. However, through repeated pairings of the neutral stimulus and the unconditioned stimulus (Figure 5.2c), the dog acquires a learned response: salivation in response to the neutral stimulus alone (Figure 5.2d). Salivation to a tone alone is an example of a conditioned response (CR). A previously neutral stimulus becomes a conditioned stimulus (CS) when it is paired with an unconditioned stimulus and begins to elicit the conditioned response. In addition to showing that salivation (CR) could be made to occur in response to a stimulus that did not naturally elicit the response, Pavlov observed that the strength of the conditioned response (the amount of salivation) increased with the number of pairings of the CS and US.

Conditioned salivation has been demonstrated with a wide variety of animals, including cats, rats, and yes, even cockroaches. Rather than using a bell or a tone as a conditioned stimulus (CS), Japanese investigators Hidehiro Watanabe and Makoto Mizunami stimulated the antennae of cockroaches with a scent of peppermint (a CS) while they placed droplets of a sugary substance (a US) on the insects’ mouths (Watanabe & Mizunami, 2007). The insects naturally salivate (a UR) to sugary substances, but not to scents applied to their antennae. After a number of repetitions of this procedure, however, the insects salivated to the scent alone (a CR). Scientists hope that by studying simpler organisms like insects, they will learn more about the mechanisms of conditioning at the neuronal level (Fountain, 2007).

We next examine other characteristics of classical conditioning: extinction and spontaneous recovery, stimulus generalization and discrimination, and stimulus characteristics that strengthen conditioned responses.

Extinction and Spontaneous Recovery

Pavlov noticed that the conditioned response of salivation to the sound of a bell or a tuning fork would gradually weaken and eventually disappear when he repeatedly presented the sound in the absence of the US (food). This process is called extinction (see Figure 5.3).

The extinguished response is not forgotten or lost to memory. It may return spontaneously at a later time when the animal is again exposed to the conditioned stimulus. This phenomenon is called spontaneous recovery. The recovered response will once again extinguish if the CS occurs in the absence of the US.

Pavlov discovered that when the CS and US are paired again after extinction has occurred, the response is likely to be learned more quickly than in the original conditioning. In many cases, the animal needs only one or two pairings. The process of relearning a conditioned response after extinction is called reconditioning.

Stimulus Generalization and Stimulus Discrimination
Pavlov found that once animals were trained to salivate to a particular stimulus, such as a particular sound, they would also salivate, but less strongly, to a related sound that varied along a continuum, such as pitch. A sound with a higher or lower pitch than the original one might elicit some degree of salivation. The tendency of stimuli similar to the conditioned stimulus to elicit a conditioned response is called a stimulus generalization.

Generally speaking, the greater the difference between the original stimulus and the related stimulus, the weaker the conditioned response. Were it not for a stimulus generalization, the animal would need to be conditioned to respond to each stimulus no matter how slightly it varied from the original conditioned stimulus.

Stimulus generalization has survival value. It allows us to respond to a range of stimuli that are similar to an original threatening stimulus. Perhaps you were menaced or bitten by a large dog when you were young. Because of stimulus generalization, you may find yourself tensing up whenever you see a large dog approaching. Not all large dogs are dangerous, of course, but stimulus generalization helps prepare us just in case.

Have you ever walked into a room and suddenly felt uncomfortable or anxious for no apparent reason? Your emotional reaction may be a conditioned response to generalized stimuli in the environment that are similar to cues associated with unpleasant experiences in the past. Perhaps, too, you have experienced déjà-vu – a feeling of having been in a place before when you’ve never actually been there. Stimulus generalization provides an explanation of these experiences. The feeling of familiarity in novel situations may be a conditioned response evoked by cues or stimuli in these novel environments that resemble conditioned stimuli encountered in other situations. A fleeting odor, the way light bounces off a ceiling, even the color of walls – all may be cues that evoke conditioned responses acquired in other settings.

Stimulus discrimination, the ability to differentiate among related stimuli, represents the opposite side of the coin to the stimulus generalization. This ability allows us to fine-tune our responses to the environment. Suppose, for example, an animal in a laboratory study receives a mild shock shortly after exposure to a CS (a tone) (Domjan, 2005). After a few pairings of the tone and shock, the animal shows signs of fear (e.g., cowering, urinating) to the tone alone. The tone is the CS, the shock in the US, and the pairing of the two leads to the acquisition of the conditioned response (CR) of fear to the tone alone. Now, let’s say that pairings of the tone and the shock continue but are interspersed with a tone of a higher pitch that is not accompanied by a shock. The animal learns to discriminate between the two stimuli, responding with fear to the original tone but remaining calm when the higher-pitched tone is sounded.

Stimulus discrimination in daily life allows us to differentiate between threatening and non-threatening stimuli. For example, through repeated noneventful encounters with certain breeds of dog, we may learn to respond with fear to a large dog of an unfamiliar breed but not to the friendly Labrador that lives next door.

Figure 5.4 illustrates the process of stimulus generalization and stimulus discrimination.

Higher-Order Conditioning

In higher-order conditioning, a new stimulus becomes a conditioned stimulus when it is paired with an established conditioned stimulus that already elicits the conditioned response (see Figure 5.5). What is learned is the association between two conditioned stimuli, or a CS-CS connection. Consider, for example, a couple that has a favorite song that was previously associated with positive feelings they had towards one another when they first met or fell in love. The songs become a conditioned stimulus (CS) that elicits these positive feelings (the CR). Other cues associated with the song, such as the jingle associated with the radio station that regularly played the song or even the name of the singer, may become conditioned stimuli that elicit a similar response. Concept Chart 5.1 presents an overview of the major concepts in classical conditioning.

Stimulus Characteristics That Strengthen Conditioned Responses

Psychologists have identified several key factors relating to the timing and intensity of stimuli that serve to strengthen conditioned responses:

1. Frequency of pairings. Generally, the most often the CS is paired with the US, the stronger and more reliable the CR will be. In some cases, however, even a single pairing can produce a strong CR. An airline passenger who experiences a sudden descent during a flight may develop an immediate and enduring fear of flying.
2. Timing. The strongest CRs occur when the CS is presented first and remains present throughout the administration of the US. Weaker CRs develop when the CS is presented first but is withdrawn before the US is introduced. Other timing sequences, such as the simultaneous presentation of the CS and US, produce even weaker CRs, if any at all.
3. Intensity of the US. A stronger US will typically lead to faster conditioning than a weaker one. For example, a puff of air (US) may be delivered shortly after a CS (e.g., a tone or a visual stimulus such as a light) is presented. The air puff produces a reflexive eyeblink response (UR). After a few pairings, a conditioned eyeblink (CR) occurs in response to the CS (tone or light) alone. A stronger air puff will lead to faster conditioning than a weaker one.

Concept chart 5.1 Key Concepts in Classical Conditioning

Concept	Description	Example: Fear of Dentistry
Classical conditioning	The process of learning by which a response identical or similar to one originally elicited by an unconditioned stimulus (US) occurs following a conditioned stimulus (CS) as the result of the prior pairing of the two stimuli	The association of pain during dental procedures with environmental stimuli in the dentist’s office leads to a learned fear response to the environmental cues alone.
Extinction	Gradual weakening and eventual disappearance of the conditioned response (CR) when the CS is repeatedly presented without the US	The use of anesthetics and painless dental techniques leads to a gradual reduction and elimination of fear of dentistry.
Spontaneous recovery	Spontaneous return of the CR some time after extinction occurs	Fear of dentistry returns spontaneously a few months or a few years after extinction.
Stimulus generalization	CR evoked by stimuli that are similar to the original CS	Person shows a fear of response when visiting the office of a new dentist.
Stimulus discrimination	CR not evoked by stimuli that are related but not identical to the CS	Person shows a fear of response to the sight of a dentist’s drill but not to equipment used for cleaning teeth.

A Cognitive Perspective on Classical Conditioning

Psychologist Robert Rescorla (1967, 1988, 2009) takes a cognitive perspective in explaining classical conditioning. He challenged the conventional behaviorist view that classical conditioning is based simply on repeated pairing of a previously neutral stimulus and an unconditioned stimulus. He argued that conditioning is a cognitive process by which the organism learns that a conditioned stimulus is a reliable signal for predicting the occurrence of the unconditioned stimulus. The tone or bell in Pavlov’s experiments with dogs creates an expectancy that food is about to arrive, which in turn leads to salivation and other feeding behaviors, such as tail wagging. To Rescorla, humans and other animals actively seek information that helps them make predictions about important events in their environments. Conditioned stimuli are signals or cues organisms use to make these predictions. The more reliable the signal, the stronger the conditioned response.

Rescorla’s model has important survival implications. Dogs and other animals may be more likely to survive if they learn to respond with salivation to cues that food is present, since salivation prepares them to swallow food. Animals are also more likely to survive if they learn to respond with fear (heightened bodily arousal) to cues that reliably signal the presence of threatening stimuli. Consider an animal that hears a sound or gets a whiff of an odor (a CS) previously associated with the presence of a particular predator (a US). By responding quickly with heightened arousal to such a stimulus, the animal is better prepared to take defensive action if the predator appears. Thus, we can think of classical conditioning as a kind of built-in early-warning system.

Rescorla’s model also explains why you are likely to develop a fear of dentistry more quickly if you experience pain during each dental visit than if you have pain only every now and then. In other words, the more reliably the CS (dental cues) signals the occurrence of the US (pain), the stronger the conditioned response is likely to be.

Why It Matters: Examples of Classical Conditioning

Pavlov’s studies might merit only a footnote in the history of psychology if classical conditioning were limited to the salivary responses of dogs. However, classical conditioning helps us explain such diverse behaviors as phobias, drug cravings, and taste aversions. John B. Watson, the founder of behaviorism, believed that Pavlov’s principles of conditioning could explain emotional responses in humans. In 1919, Watson set out with Rosalie Rayner, a student who was later to become his wife, to prove that fear response could be acquired through classical conditioning. After taking a look at Watson and Rayner’s experiment, we consider other examples in humans.

Classical Conditioning of Fear Responses

As their subject, Watson and Rayner selected an 11-month-old boy whom they called Albert B., but who is better known in the annals of psychology as Little Albert (Watson & Rayner, 1920). Albert had previously shown no fear of a white rat that was placed near him and had even reached out to stroke the animal (see Figure 5.6). In the experimental procedure, the rat was placed close to Albert, and as he reached for it, Watson banged a steel bar with a hammer just behind his head, creating a loud gong. Watson believed that loud sounds naturally make infants cringe and shudder with fear. Sure enough, Albert showed signs of fear when the bar was struck – crying and burying his face in the mattress. Watson and Rayner then repeatedly paired the rat and the loud sound, which resulted in Albert’s developing a fear response to the sight of the rat alone. Such an acquired fear response is called a conditioned emotional reaction (CER). Later experiments showed that Albert’s fear response had generalized to other furry stimuli, including a dog, a rabbit, and even a Santa Claus mask that Watson had worn.

Let us examine the Watson and Rayner study by applying what we know about classical conditioning. Before conditioning, Albert showed no fear of the white rat; it was a neutral stimulus. The unconditioned stimulus (US) was the loud banging sound, a stimulus that naturally elicits a fear response (UR) in young children. Through repeated pairings of the white rat and the banging sound (US), the white rat alone (CS) came to elicit a fear response (CR).

Though the Little Albert experiment is among the most famous studies in psychology, it would not pass muster with the stricter ethical standards in place today. Exposing a child to intense fear, even with the parents’ permission, fails to adhere to the responsibility investigators have to safeguard the welfare of research participants. In addition, Watson and Rayner made no attempt to undo or extinguish Albert’s fears, as ethical codes would now require, although they did discuss techniques they might use to do so. We can’t say for sure what became of Little Albert, but investigators report that, sadly, the child they believe may have been Albert succumbed to a childhood illness at the age of six (H. P. Beck, Levinson, & Irons, 2009).

Excessive fears, or phobias, such as Albert’s fear of white rats or the fear of dentistry, can be acquired through classical conditioning (Field, 2006; J. J. Kim & Jung, 2006). In one case example, a 34-year-old woman had been terrified of riding on elevators ever since a childhood incident in which she and her grandmother were trapped on an elevator for hours. For her, the single pairing of previously neutral stimuli (cues associated with riding on elevators) and a traumatic experience was sufficient to produce an enduring phobia (fear of elevators). In some cases, the original conditioning experiences may be lost to memory, or have occurred even before language developed (as in Albert’s case).

Early work on classical conditioning of fear responses set the stage for the development of behavior therapy, a form of therapy involving the systematic application of principles of learning to help people overcome phobias and change problem behaviors such as addictive behaviors and childhood behavior problems. We discuss specific applications of behavior therapy in Chapter 16.

Classical Conditioning of Positive Emotions

It’s not only negative emotions like fear that can be classically conditioned. Perhaps you’ve had the experience of hearing a certain song on the radio and suddenly smiling or feeling cheerful, or even experiencing a tinge of sexual arousal. Chances are the song evoked past experiences associated with pleasant emotions or sexual arousal. Similarly, feelings of nostalgia may be conditioned responses elicited by subtle cues in the environment that had come to be associated with pleasant experiences in the past. These cues – perhaps just a whiff of perfume or the mist in the air on a spring day – may induce nostalgic feelings.

Classical Conditioning of Drug Cravings

People with chemical dependencies frequently encounter drug cravings, especially when they undergo drug withdrawal or go “cold turkey.” Though cravings may have a physiological basis (they constitute part of the withdrawal syndrome for addictive drugs), classical conditioning can also contribute to these strong desires. Cravings may be elicited by cues (conditioned stimuli) in the environment associated with prior drug use. People battling alcoholism may experience strong cravings for the drug when they are exposed to drug-related conditioned stimuli, such as the sight of a bottle of alcohol. Cravings may be elicited by conditioned stimuli long after withdrawal symptoms have passed, such as cues associated with a subway station where a drug abuser formerly bought drugs.

The conditioning model of drug cravings is supported by research that demonstrates that people with alcoholism salivate more at the sight and odor of alcohol than do non-alcoholic people (Monti et al., 1987). Salivating to the sound of a tone may be harmless enough, but salivating when looking at a picture of a Scotch bottle in a magazine can be dangerous to a person struggling with alcoholism. Not surprisingly, drug counselors encourage recovering drug and alcohol abusers to avoid cues associated with their former drug-use patterns.

Classical Conditioning of Taste Aversions

The principles of classical conditioning can also be used to explain conditioned taste aversions, including my disgust for eggs (Ferreira et al., 2006; Garcia & Koelling, 2009; Limebeer & Parker, 2006). Psychologist John Garcia was the first to demonstrate experimentally the role of classical conditioning in the acquisition of taste aversions. Garcia and colleague Bob Koelling noticed something unusual in the behavior of rats that had been exposed to nausea-inducing radiation. The rats developed an aversion or “conditioned nausea” to flavored water sweetened with saccharin when the water was paired with the nausea-producing radiation (Garcia & Koelling, 1966). In classical conditioning terms, the radiation was the US; the nausea it produced was the UR.; the flavored water was the CS; and the aversion (nausea) the CS elicited on its own was the CR.

In related work, Garcia demonstrated that aversion to particular foods could be classically conditioned by giving rats a nausea inducing drug soon after they ate the foods (Garcia & Koelling, 1971). Moreover, taste aversions were acquired even when the CS (the taste of the food) was presented a few hours before the presentation of the US (the nausea-inducing stimulus) (Domjan, 2005). This discovery shocked Garcia’s experimental colleagues, who believed that classical conditioning could occur only when the CS is followed almost immediately by the US. Moreover, Garcia and his team were able to demonstrate that conditioned taste aversions could be acquired on the basis of a single pairing of the flavor of a food or drink with a nausea-inducing stimulus.

Like other forms of classical conditioning, conditioned taste aversions have clear survival benefits. Our ancestors lived without the benefit of refrigeration or preservatives. Acquiring an aversion to foods whose rancid smells and tastes sickened them would have helped them avoid such foods in the future. In a classic study that literally applied the principles of classical conditioning on the range, John Garcia and his colleagues came up with an ingenious way to help sheep ranchers protect their sheep from coyotes (Gustavson & Garcia, 1974; Gustavson et at, 1974). At the time of the study, free-ranging coyotes were killing thousands of sheep, and ranchers seeking to protect their flocks were killing so many coyotes that their survival as a species was endangered. It was therefore important to find a way of stopping the coyotes’ destructive behavior without killing them. As an experiment, the researchers injected sheep carcasses with a poison that would sicken but not kill the coyotes, and scattered the carcasses over the range. Not only did sheep killings decline, but some coyotes developed such an aversion to the taste of sheep meat that they ran away just at the sight or smell of sheep.

Conditioning the Immune System

In a landmark study, Robert Ader and Nicholas Cohen (1982) showed that classical conditioning even extends to the workings of the immune system. The immune system protects the body from disease-causing organisms. The researchers had laboratory rats ingest saccharin-sweetened water (CS) while simultaneously giving them a drug (US) that suppressed immune system responses (UR). After several pairings of the CS and US, immune suppression (CR) occurred when the rats drank the sweetened water alone (CS).

Conditioned immune suppression can be made to occur in response to other conditioned stimuli, such as odors and sounds, as well as in humans (Kusnecov, 2001; Pacheco-Lopez, et al., 2005). For example, a group of healthy people were an immune-suppressant drug as an unconditioned stimulus, which was paired with a distinctively flavored drink as a conditioned stimulus during four separate sessions spread over three days (Goebel et al., 2002). Afterward, presenting the drink without the active drug succeeded in suppressing immune-system response, thereby demonstrating the acquisition of a conditioned response.

The ability to acquire an immune-suppressant response through classical conditioning may have important health implications for humans. In people who receive organ transplants, the immune system attacks the transplanted organs as foreign objects. Perhaps classical conditioning can be used to suppress the tendency of the body to reject transplanted organs, lessening the need for immune-suppressant drugs. It is also conceivable that classical conditioning may be used to give the immune system a boost in its fight against disease, perhaps even to strength the body’s ability to defend itself against cancer. However, we need further research to determine the value of classical conditioning in medical treatment.

• What is Thorndike’s Law of Effect?
• What is operant conditioning?
• What are the different types of reinforcers?
• What are the different schedules of reinforcement, and how are they related to learning?
• What are escape learning and avoidance learning?
• What is punishment, and why are psychologists concerned about its use?
• What are some applications of operant conditioning?

Classical conditioning can explain how we learn relatively simply, reflexive response, such as salivation and eyeblinks, as well as emotional responses associated with fear and disgust. But classical conditioning cannot explain how we learn the more complex behaviors that are part and parcel of our experiences: getting up in the morning, dressing, going to work or school, preparing meals, taking care of household chores, running errands, completing homework, and socializing with friends, among countless other behaviors of daily life. To account for the process of learning such complex behaviors, we turn to a form of learning called operant conditioning. With classical conditioning, learning results from the association between stimuli before a response occurs. With operant conditioning, learning results from the association of a response with its consequences. With operant conditioning, responses are acquired and strengthened on the basis of the effects they produce in the environment.

Our study of operant conditioning focuses on two American psychologists: Edward Thorndike, whose Law of Effect was the first systematic attempt to describe how behavior is affected by its consequences, and B. P. Skinner, whose experimental work laid out many of the principles of operant conditioning.

Thorndike and the Law of Effect

Edward Thorndike (1874-1947) studied learning in animals because he found them easier to work with than people. He constructed a device called a “puzzle box” – a cage in which the animal (usually a cat) had to perform a simple act (such as pulling a looped string or pushing a pedal) in order to make its escape and reach a dish of food placed within its view just outside the cage (see Figure 5.7). The animal would first engage in seemingly random behaviors until it accidentally performed the response that released the door. Thorndike argued that the animals did not employ reasoning, insight, or any other form of higher intelligence to find their way to the exit. Rather, through a random process of trial and error they gradually eliminated useless responses and eventually chanced upon the successful behavior. Successful responses were then “stamped in” by the pleasure they produced and became more likely to be repeated in the future.

Based on his observations, Thorndike (1905) proposed a principle that he called the Law of Effect, which holds that the tendency for a response to occur depends on the effects it has on the environment (P. L. Brown & Jenkins, 2009). Specifically, Thorndike’s Law of Effect states that responses that have satisfying effects are strengthened and become more likely to occur again in a given situation, whereas responses that lead to discomfort are weakened and become less likely to recur. Modern psychologists call the first part of the Law of Effect reinforcement and the second part punishment (L. T. Benjamin, 1988).

Thorndike went on to study how the principles of animal learning that he formulated could be applied to human behavior and especially to education. He believed that although human behavior is certainly more complex than animal behavior, it, too, can be explained on the basis of trial-and-error learning in which accidental successes become “stamped in” by positive consequences.

B. F. Skinner and Operant Conditioning

Thorndike laid the groundwork for an explanation of learning based on the association between responses and their consequences. It would fall to another American psychologist, B. F. Skinner (1904-1990), to develop a more formal model of this type of learning, which he called operant conditioning.

Skinner was arguably not only the most famous psychologist of his time but also the most controversial. What made him famous was his ability to bring behaviorist principles into the public eye through his books, articles in popular magazines, and public appearances. What made him controversial was his belief in radical behaviorism, which holds that behavior, whether animal or human, is completely determined by environmental and genetic influences. Free will, according to Skinner, is but an illusion or a myth. Though the staunch behaviorism he espoused was controversial in his own time and remains so today, there is no doubt that his concept of operant conditioning alone merits him a place among the pioneers of modern psychology.

Like Watson, Skinner was a strict behaviorist who believed that psychologists should limit themselves to the study of observable behavior. Because “private events,” such as thoughts and feelings, cannot be observed, he believed they have no place in a scientific account of behavior. For Skinner, the mind was a “black box” whose contents cannot be illuminated by science.

Skinner allowed that some responses occur reflexively, as Pavlov had demonstrated. But classical conditioning is limited to explaining how new stimuli can elicit existing behaviors, such as salivation. It cannot account for new behaviors, such as the behavior of the experimental animals in Thorndike’s puzzle box. Skinner found in Thorndike’s work a guiding principle that behavior is shaped by its consequences. However, he rejected Thorndike’s mentalistic concept that consequences influence behavior because they produce “satisfying effects.” Skinner proposed that organisms learn responses that operate on the environment to produce consequences; he therefore called this learning process operant conditioning.

Skinner studied animal learning using a device we now call a Skinner box. The Skinner box is a cage that contains a food-release mechanism the animal activates when it responds in a certain way – for example, by pressing a lever or pushing a button.

Through operant conditioning, organisms learn responses, such as pressing a bar, that produce changes in the environment (release of food). In this form of learning, the consequences of a response determine the likely-hood that the response will occur again. The response itself is called an operant response or, more simply, an “operant.” Behaviors that produce rewarding effects are strengthened – that is, they become more likely to occur again. In effect, a well-trained operant response becomes a habit (Staddon & Cerutti, 2003). For example, if your teacher responds to a question only if you first raise your hand, you will become more likely to develop the habit of raising your hand before asking a question.

Operant conditioning is also called instrumental learning since the behavior is instrumental in bringing about rewarding consequences. The term reinforcer refers to a stimulus or event that increases the likelihood that the behavior it follows will be repeated. For example, the act of answering questions when students raise their hands is a reinforcer.

Skinner observed that the longer reinforcement is delayed, the weaker its effects will be. A rat or a pigeon in the Skinner box, or a child in the classroom, will learn the correct responses faster when reinforcement follows the response as quickly as possible. In general, learning progresses more slowly as the delay between response and reinforcement increases. Skinner also showed how operant conditioning could explain some forms of superstitious behavior. Consider a baseball player who hits a home run after a long slump and then wears the same pair of socks he had on at the time for good luck in every remaining game of the season. The superstitious behavior could be understood in terms of mistaking a mere coincidence between a response (wearing a particular pair of socks) and a reinforcement (home run) for a connection between the two.

Many commonly held superstitions – from not stepping on cracks in the sidewalk to throwing salt over one’s shoulder for good luck – are part of our cultural heritage, handed down from generation to generation. Perhaps there was a time when these behaviors were accidentally reinforced, but they have become so much a part of our cultural tradition that people no longer recall their origins.

In the next sections, we review the basic principles of operant conditioning.

Principles of Operant Conditioning

Experimental work by Skinner and other psychologists established the basic principles of operant conditioning, including those we consider here: positive and negative reinforcement, primary and secondary reinforcers, discriminative stimuli, shaping, and extinction.

Positive and Negative Reinforcement

Skinner distinguished between two types of reinforcement, positive reinforcement and negative reinforcement. In positive reinforcement, a response is strengthened by the introduction of a stimulus after the response occurs. This type of stimulus is called a positive reinforcer or reward. Examples of positive reinforcers include food, money, and social approval. You are more likely to continue working at your job if you receive a steady paycheck (a positive reinforcer) than if the checks stop coming. You are more likely to study hard for exams if your efforts are rewarded with good grades (another positive reinforcer) than if you consistently fail (see Figure 5.8).

In negative reinforcement, a response is strengthened when it leads to the removal of an “aversive” (unpleasant or painful) stimulus. Negative reinforcers are aversive stimuli such as loud noise, cold, pain, nagging, or a child’s crying. We are more likely to repeat behaviors that lead to their removal. A parent’s behavior in picking up a crying baby to comfort it is negatively reinforced when the baby stops crying; in this case, the aversive stimulus of crying has been removed. Many people are confused about the meaning of negative reinforcement because the term “negative” implies punishment (Chance, 2009). However, remember that any form of reinforcement, whether positive or negative, actually strengthens behavior.

The difference is that in positive reinforcement, behaviors are strengthened when they are followed by the introduction or presentation of a stimulus, whereas in negative reinforcement, behaviors are strengthened when they lead to the removal of a stimulus. A positive reinforcer is typically a rewarding stimulus (e.g., food or praise), whereas a negative reinforcer is typically an unpleasant or aversive stimulus (e.g., pain or crying).

Negative reinforcement can be a two way street. Crying is the only means infants have of letting us know when they are hungry or wet or have other needs. It is also an aversive stimulus to anyone within earshot. It is a negative reinforcer because parents will repeat behaviors that succeed in stopping the infant’s crying. The baby’s crying is positively reinforced by the parents’ responses. (Like Skinner’s pigeons, parents may need to do some “pecking around” to find out what Junior wants: “Let’s see, he’s not wet, so he must be hungry.”)

Negative reinforcement may have undesirable effects in some situations. Consider a child who throws a tantrum in a toy store when the parent refuses the child’s request for a particular toy. The child may have learned from past experience that tantrums get results. In operant learning terms, when a tantrum does get results, the child is positively reinforced for throwing the tantrum (because the parent “gives in”), while the parent is negatively reinforced for complying with the child’s demands because the tantrum stops. Unfortunately, this pattern of reinforcement only makes the recurrence of tantrums more likely.

Primary and Secondary Reinforcers

At 16 months of age, my daughter Daniella became intrigued with the contents of my wallet. It wasn’t those greenbacks with the pictures of Washington and Lincoln that caught her eye. No, she ignored the paper money but was fascinated with the holograms on the plastic credit cards. The point here is that some stimuli, called primary reinforcers, are intrinsically rewarding because they satisfy basic biological needs or drives. Their reward or reinforcement value does not depend on learning. Primary reinforcers include food, water, sleep, relief from pain or loud noise, oxygen, sexual stimulation, and novel visual stimuli, such as holograms.

Other reinforcers, called secondary reinforcers, acquire their reinforcement value through a learning process by which they become associated with primary reinforcers (Chance, 2009). Money is a secondary reinforcer (also called a conditioned reinforcer). It acquires reinforcement value because we learn it can be exchanged for more basic reinforcers, such as food or clothing. Other examples of secondary reinforcers include good grades, awards, recognition, smiles, and praise. Much of our daily behavior is influenced by secondary reinforcers in the form of expressions of approval from others.

Discriminative Stimuli

Let’s say you put a rat in a Skinner box and reinforce it with food when it presses a bar, but only if it makes that response when a light is turned on (see Figure 5.9). When the light is off, it receives no reinforcement no matter how many times it presses the bar. How do you think the rat will respond? Clearly, the rate of response will be much higher when the light is on than when it is off. The light is an example of a discriminative stimulus, a cue that signals that reinforcement is available if the subject makes a particular response.

Our physical and social environment is teeming with discriminative stimuli. When is the better time to ask someone for a favor: When the person appears to be down in the dumps or is smiling and appears cheerful? You know the answer. The reason you know is that you have learned that a person’s facial cues serve as discriminative stimuli that signal times when requests for help are more likely to be positively received. You also know that your professors are more likely to respond to your raising your hand if they are facing you than if their backs are turned. A green traffic light, another type of discriminative stimulus, signals that driving through an intersection is likely to be reinforced by a safe passage.

Shaping

Rats don’t naturally press levers or bars. If you place a rat in a Skinner box, it may eventually happen upon the correct response through trial and error. However, the experimenter can hasten the learning process by using the technique of shaping. Shaping involves learning in small steps through applying the method of successive approximations, in which the experimenter reinforces a series of ever-closer approximations of the target response (Kreuger & Dayan, 2009). The experimenter may at first reinforce the rat when it moves to the part of the cage that contains the bar. Once this behavior is established, reinforcement occurs only if the animal moves closer to the bar, then closer still, then touches the bar with its paw, and then actually presses the bar. If you have ever observed animal trainers at work, you will recognize how shaping is used to train animals to perform a complex sequence of behaviors.

We put the method of successive approximations into practice in our daily lives when we attempt to teach someone a new skill, especially one involving a complex set of behaviors. When teaching a child to swim, the instructor may deliver verbal reinforcement (telling the child he or she is doing “great”) each time the child successfully performs a new step in the series of steps needed to develop proper form.

Extinction

You’ll recall from Module 5.1 that extinction of classically conditioned responses occurs when the conditioned stimulus is repeatedly presented in the absence of the unconditioned stimulus. Similarly, in operant conditioning, extinction is the process by which responses are weakened and eventually eliminated when the response is repeatedly performed but is no longer reinforced. Thus, the bar-pressing response of a rat in the Skinner box will eventually be extinguished if reinforcement (food) is withheld. If you repeatedly raise your hand in class but aren’t called upon, you will probably in time stop raising your hand.

Schedules of Reinforcement

In the Skinner box, an animal can be reinforced for each peck or bar press, or for some portion of pecks or bar presses. One of Skinner’s major contributions was to show how these different schedules of reinforcement – predetermined plans for timing the delivery of reinforcement – influence learning.

In a schedule of continuous reinforcement, reinforcement follows each instance of the operant response. The rat in the Skinner box receives a food pellet every time it presses the lever. Similarly, if a light comes on every time you flick a light switch, you will quickly learn to flick the switch each time you enter a darkened room. Operant responses are learned most rapidly under a schedule of continuous reinforcement. However, continuous reinforcement also leads to rapid extinction when reinforcement is withheld. How long will it take before you stop flicking the light switch if the light fails to come on because the bulb needs replacing? Just one or two flicks of the switch without results may be sufficient to extinguish the response. But extinction does not mean the response is forgotten or lost to memory. It is likely to return quickly once reinforcement – that is, once you install a new bulb.

Responses are more resistant to extinction under a schedule of partial reinforcement than under a schedule of continuous reinforcement. In a schedule of partial reinforcement, only a portion of responses is reinforced. Because this makes it more unlikely that an absence of reinforcement will be noticed, it takes longer for the response to fade out.

In daily life, schedules of partial reinforcement are much more common than schedules of continuous reinforcement. Think what it would mean to be reinforced on a continuous basis. You would receive a reinforce (reward) each time you come to class, cracked open a textbook, or arrived at work on time. However desirable this rate of reinforcement might seem, it is no doubt impossible to achieve day to day. Fortunately, partial-reinforcement schedules produce overall high response rates and have the added advantage of greater resistance to extinction.

Partial reinforcement is administered under two general kinds of schedules: ratio schedules and interval schedules. In ratio schedules, reinforcement is based on the number of responses. In interval schedules, reinforcement is based on the timing of responses. For each type, reinforcement can be administered on either a fixed or variable basis.

Let’s break this down further. Figure 5.10 shows typical rates of response under different schedules of partial reinforcement. Notice how much faster response rates are under ratio schedules than under interval schedules. To account for this difference, remember that in ratio schedules, reinforcement depends on the number of responses, not on the length of time elapsed since the last reinforcement, as is the case with interval schedules.

Fix-Ratio (FR) Schedule

In a fixed-ratio (FR) schedule, reinforcement is given after a specified number of correct responses. For example, in an “FR-6” schedule, reinforcement is given after each sixth response. The classic example of fixed-ratio schedule is piecework, in which workers are paid on the basis of how many items they produce. Fixed-ratio schedules produce a constant, high level of response, with a slight dip in responses occurring after each reinforcement (see Figure 5.10). On a fixed-ratio schedule, the faster people work, the most items they produce and the most money they earn. However, quality may suffer if quantity alone determines how reinforcements are dispensed.

Variable-Ratio (VR) Schedule

In a variable-ratio (VR) schedule, the number of correct responses needed before reinforcement is given varies around an average value. For example, a “VR-20” schedule means that reinforcement is administered after an average of every 20 responses. In some cases, reinforcement may be delivered after two, five, or 10 responses; at other times, 30 or 40 responses may be required.

Salespeople who make “cold calls” to potential customers are reinforced on a variable ratio schedule. They may average 20 or 30 calls to make a sale, but they don’t know how many calls it will take to make a sale on any given day. Gambling is also reinforced on a variable-ratio schedule, as you never know how many bets it will take to produce a winner. When playing a slot machine, for example, a win (reinforcement) may occur after one, two, 10, or even 50 or more tries. (Not wonder it’s called a one-armed bandit.)

Variable-ratio schedules typically produce high and steady rates of response (see Figure 5.10). They are also more resistant to extinction than fixed-ratio schedules because we never known when a given response will be rewarded. This may explain why so many people routinely buy weekly lottery tickets, even though they may win only piddling amounts every now and then. As an advertisement for one state lottery put it, “Hey, you never know.”

Fixed-Interval (FI) Schedule

In a fixed-interval (FI) schedule, reinforcement is given only for a correct response made after a fixed amount of time has elapsed since the last reinforcement. On an “FI-30” schedule, for example, an animal in a Skinner box receives a food pellet if it makes the required response after an interval of 30 seconds has elapsed since the last food pellet was delivered, regardless of the number of responses it made during the 30-second interval. Workers who receive a regular paycheck at the end of every week or two are reinforced on a fixed-interval schedule. Fixed-interval schedules tend to produce a “scalloped” response pattern in which the rate of response tends to dips just after reinforcement is given and then increases as the end of the interval approaches (see Figure 5.10). An example of the scalloping effect occurs when workers who receive monthly performance reviews show more productive behaviors in the days leading up to their evaluations than immediately afterward.

Variable-Interval (VI) Schedule

In a variable-interval (VI) schedule, the amount of time that must elapse before a reinforcement is given for a correct response varies rather than remaining fixed. In laboratory studies, a “VI-60” schedule means that the period of time that must elapse before reinforcement is given varies around an average of 60 seconds. On any given occasion, the interval could be as short as one second or as long as 120 seconds, but the average across occasions remains 60 seconds. Trying to hail a cab on a city street is a real-life example; it may take only a minute or two on some occasions, but much (much) longer on other occasions.

Teachers who spring pop quizzes in class are using a variable-ratio schedule of reinforcement to encourage regular studying. Because students never know the actual days that quizzes are given, they are more likely to be reinforced by receiving better grades if they prepare for each class by studying regularly. With scheduled tests, reinforcement is based on a fixed-interval schedule; that is, rewards for studying become available only at the regular times the tests are given. In this case, we expect a scalloped rate of response that is typical of fixed-interval reinforcement – an increased rate of studying, perhaps cramming, just before the test and a drop in studying immediately afterward. Radio stations try to get you to tune in regularly by periodically offering prizes on a variable schedule of reinforcement. You never know when they might announce, “Be the fifth caller and you’ll receive free tickets to....”

Variable-interval schedules tend to produce a slow but steady rate of response. They also tend to be more resistant to extinction than fixed-interval schedules, because one never knows when reinforcement may occur.

Escape Learning and Avoidance Learning

In escape learning, an organism learns to escape an aversive stimulus by performing an operant response. The escape behavior is negatively reinforced by the removal of the aversive or painful stimulus. A rat may be taught to press a bar to turn off an electric shock. We may learn to escape from the heat of a summer day by turning on a fan or air conditioner.

In avoidance learning, the organism learns to perform a response that avoids an aversive stimulus. The rat in a Skinner box may receive a signal (e.g., a tone) that a shock is about to be delivered. The animal learns to avoid the shock by performing the correct response, such as pressing a bar. You open an umbrella before stepping out in the rain to avoid the unpleasant experience of being drenched.

Like other forms of learning, escape learning and avoidance learning may be adaptive in some circumstances but not in others. We learn to apply sunscreen to avoid sunburn, which is adaptive. But skipping regular dental visits to avoid unpleasant or painful dental procedures is not, as it can lead to more serious dental problems or even to tooth loss. People may turn to alcohol or other drugs to escape from their problems or troubling emotions. But the escape is short lived, and problems resulting from drug or alcohol abuse can quickly compound the person’s initial difficulties.

You may at this point wish to review the key concepts in operant conditioning outlined in Concept Chart 5.2.

Punishment

Punishment is the flip side of reinforcement: It weakens the behavior it follows, whereas reinforcement strengthens the preceding behavior. Just as there are positive and negative forms of reinforcement, there are also positive and negative forms of punishment. In positive punishment, an aversive or unpleasant stimulus is imposed as a consequence of an undesirable behavior, which over time tends to reduce the frequency of the undesirable behavior. Examples include a parent who scolds or spanks a child who “talks back,” or the imposition of penalties in the form of monetary fines for speeding or illegal parking.

In negative punishment, a reinforcing stimulus is removed as a consequence of an undesirable behavior, which over time tends to reduce the frequency of the undesirable behavior. Examples include turning off the TV when a child misbehaves, taking away privileges (e.g., grounding teenagers), or removing the misbehaving child from a reinforcing environment (a “time-out”). The use of punishment, whether positive or negative, tends to reduce the frequency of the behavior it follows.

Let’s clarify the difference between two terms which mean very different things but are frequently confused, negative reinforcement and punishment. Behavior that leads to the removal of an unpleasant or aversive stimulus is negatively reinforced and becomes stronger as a result, such as fastening the seat belt in the car to turn off that annoying dinging sound. Behavior that leads to the imposition of an unpleasant stimulus (or removal of a desirable stimulus) is punished and becomes weaker as a result, such as driving over the speed limit after receiving a moving violation (or two) (see Figure 5.11).

CONCEPT CHART 5.2 Key Concepts in Operant Conditioning
Concept	Description	Example
Nature of operant conditioning	A form of learning in which responses are strengthened by the effects they have in the environment	If students receive answers to their questions only when they raise their hands before asking them, hand-raising behavior is strengthened
Discriminative stimulus	A stimulus that indicates that reinforcement will be available if the correct response is made	A child learns to answer the phone when it rings and to wait for a dial tone before dialing
Positive reinforcer	A stimulus or event that makes the response it follows more likely to occur again	Praising children for picking up their clothes increases the likelihood that they will repeat the behavior
Negative reinforcer	An aversive stimulus whose removal strengthens the preceding behavior and increases the probability that the behavior will be repeated	The annoying sound of a buzzer on an alarm clock increases the likelihood that we will get out of bed to turn it off
Primary reinforcer	A stimulus that is innately reinforcing because it satisfies basic biological needs or drives	Food, water, and sexual stimulation are primary reinforcers
Secondary reinforcer	A stimulus whose reinforcement value derives from its association with primary reinforcers	Money, which can be exchanged for food and clothing, is a secondary reinforcer
Shaping	A process of learning that involves the reinforcement of increasingly closer approximations to the desired response	A boy learns to dress himself when the parent reinforces him for accomplishing each small step in the process
Extinction	The gradual weakening and elimination of an operant response when it is not reinforced	A girl stops calling out in class without first raising her hand when the teacher fails to respond to her
Schedule of continuous reinforcement	A schedule for delivering reinforcement every time a correct response is produced	A girl receives praise each time she puts her clothes away
Schedule of partial reinforcement	A schedule of delivering reinforcement in which only a portion of responses is reinforced	A boy receives praise for putting his clothes away every third time he does it (fixed-ratio schedule)
Escape learning	Learning responses that result in escape from an aversive stimulus	A motorist learns to detour to escape from congested traffic
Avoidance learning	Learning responses that result in avoidance of an aversive stimulus	A person leaves for work an hour early to avoid heavy traffic
Punishment	Imposing an unpleasant or aversive consequence (i.e., positive punishment), or withdrawing a desirable stimulus (i.e., negative punishment), in response to an undesirable behavior	The government imposes a late penalty when a taxpayer fails to file tax returns on time (positive punishment) A teen loses driving privileges for coming home after curfew (negative punishment)

Psychologists and pediatricians encourage parents not to rely on punishment as a means of disciplining their children; instead, they recommend reinforcing desirable behaviors (American Academy of Pediatrics, 1998; Gershoff, 2002a, 2002b). Punishment, especially physical punishment, has many drawbacks, including the following:

• Punishment may suppress undesirable behavior, but it doesn’t eliminate it. The punished behavior often returns when the punishing stimulus is withdrawn. For example, the child who is punished for misbehavior may perform the undesirable behavior when the parents aren’t looking.
• Punishment does not teach new behaviors. Punishment may suppress an undesirable behavior, but it does not help the child acquire a more appropriate behavior in its place.
• Punishment can have undesirable consequences. Punishment, especially physical punishment, can lead to strong negative emotions in children, such as anger, hostility, and fear directed toward the parent or other punishing agent. Fear may also generalize. Children repeatedly punished for poor performance in school may lose confidence in themselves or develop a fear of failure that handicaps their academic performance. They may cut classes, withdraw from challenging courses, or even drop out of school.
• Punishment may become abusive. Stopping a child’s undesirable behavior, at least temporarily, may reinforce the parents for using spankings or other forms of physical punishment. This may lead to more frequent physical punishment that crosses the line between discipline and abuse (Gershoff, 2002a, 2002b). Another type of abusive situation occurs when parents turn to ever-harsher forms of punishment when milder punishments fail. Abused children may harbor intense rage or resentment toward the punisher, which they may vent by responding aggressively against that person or other, less physically imposing targets, such as peers or siblings.
• Punishment may represent a form of inappropriate modeling. When children observe their parents resorting to physical punishment to enforce compliance with their demands, the lesson they learn is that using force is an acceptable way of resolving interpersonal problems.

Do the drawbacks of punishment mean that it should never be used? There may be some occasions when punishment is appropriate. Parents may need to use punishment to stop children from harming themselves or others (e.g., by running into the street or hitting other children in the playground). But parents should avoid using harsh physical punishment. Examples of milder punishments include (1) verbal reprimand (“No, Johnny, don’t do that. You can get hurt that way.”); (2) removal of a reinforcer, such as grounding teenagers or taking away a certain number of points or tokens that children receive each week that are exchangeable for tangible reinforcers (e.g., toys, special activities); and (3) time-out, or temporary removal of a child from a reinforcing environment following misbehavior.

Parents who use punishment should help the child understand why he or she is being punished. Children may think they are being punished because they are “bad.” ‘They may think negatively of themselves or fear that Mommy and Daddy no longer love them. Parents need to make clear exactly what behavior is being punished and what the child can do differently in the future. In this way, parents can help children learn more desirable behaviors. Punishment is also more effective when it is combined with positive reinforcement for desirable alternative behaviors.

Before going further, you may wish to review Table 5.1, which compares reinforcement and punishment.

Why It Matters: Applications of Operant Conditioning

In many ways, the world is like a huge Skinner box. From the time we are small children, reinforcements and punishments mold our behavior. We quickly learn which behaviors earn approval and which incur disapproval. How many thousands upon thousands of reinforcements have shaped your behavior over the years? Would you be taking college courses today were it not for the positive reinforcement you received from an early age for paying attention in class, doing your homework, studying for exams, and getting good grades? Who were the major reinforcing agents in your life? Your mother or father? A favorite uncle or aunt? Your teachers or coaches? Yourself?

Psychologists have developed a number of applications of operant conditioning, including bio-feedback training, behavior modification, and programmed instruction.

Biofeedback Training: Using Your Body’s Signals as Reinforcers

Chapter 3 introduced the topic of biofeedback training, a technique for learning to change certain bodily responses, including brain wave patterns and heart rate. Biofeedback training relies on operant conditioning principles. Physiological monitoring devices record changes in bodily responses and transmit the information to the user, typically in the form of auditory signals that provide feedback regarding desirable changes in these responses. The feedback reinforces behaviors (e.g., thinking calming thoughts) that bring about these desirable changes.

Behavior Modification: Putting Learning Principles into Practice

Behavior modification (B-mod) is the systematic application of learning principles to strengthen adaptive behavior and weaken maladaptive behavior. For example, Skinner and his colleagues applied operant conditioning principles in a real-world setting by establishing the first token economy program in a mental hospital. In a token economy program, patients receive tokens, such as plastic chips, for performing desired behaviors, such as dressing and grooming themselves, making their beds, or socializing with others. The tokens are exchangeable for positive reinforcers, such as extra privileges.

Behavior modification programs are also applied in the classroom, where they have produced measurable benefits in academic performance and social interactions, and reductions in aggressive and disruptive behaviors and truancy. Teachers may use tokens or gold stars to reward students for appropriate classroom behavior and academic achievement. Children can use the tokens or gold stars at a later time to “purchase” small prizes or special privileges, such as more recess time.

Parent training programs bring behavior modification into the home. After training in using B-mod techniques, parents reward children’s appropriate behaviors and punish noncompliant and aggressive behaviors with time-outs and suspension of privileges and rewards.

Programmed Instruction

Skinner applied operant conditioning to education in the form of programmed instruction. In programmed instruction, the learning of complex material is broken down into a series of small steps. The learner proceeds to master each step at his or her own pace. Skinner even designed a “teaching machine” that guided students through a series of questions of increasing difficulty. After the student responded to each question, the correct response would immediately appear. This provided immediate reinforcement for correct responses and allowed students to correct any mistakes they had made. Since questions were designed to build upon each other in small steps, students would generally produce a high rate of correct response and thus receive a steady stream of reinforcement. Teaching machines have since given way to computerized forms of programmed instruction, called computer-assisted instruction, in which the computer guides the student through an inventory of increasingly more challenging questions.

TABLE 5.1 Comparing Reinforcement and Punishment
Technique	What Happens?	When Does This Occur?	Example	Consequences on Behavior
Positive reinforcement	A positive event or stimulus is introduced	After a response	Your instructor smiles at you (a positive stimulus) when you answer a question correctly	You become more likely to answer questions in class
Negative reinforcement	An aversive stimulus is removed	After a response	Buckling the seat belt turns off the annoying buzzer	You become more likely to buckle your seat belt before starting the engine
Positive punishment (application of aversive stimulus)	An aversive stimulus is applied	After a response	A parent scolds a child for slamming a door	The child becomes less likely to slam doors
Negative punishment (removal of a reinforcing stimulus)	A reinforcing stimulus is removed	After a response	A child loses TV privileges for hitting a sibling	The child becomes less likely to engage in hitting