Rocky Balboa just punched me: The neuroscience behind our tears, fears and flinches at the movies

We know we're at the movies, but sometimes jump, bawl and squeal anyway — and genius brain science explains why

Published January 18, 2015 8:30PM (EST)

Sylvester Stallone as Rocky Balboa in "Rocky IV"       (MGM Studios, Inc.)
Sylvester Stallone as Rocky Balboa in "Rocky IV" (MGM Studios, Inc.)

Excerpted from "Flicker: Your Brain On Movies"

"Rocky II." It’s round 15 of the rematch between Rocky Balboa and Apollo Creed. The two fighters are beating each other into hamburger. As the final round builds, the camera pulls in close to the fighters. Creed delivers a series of haymakers with his right, and Balboa answers with punishing alternating body blows. These are the punches that in a few moments will bring both fighters to the mat. Most of the shots are over-the-shoulder views from the perspective of the boxer receiving the blows. So as you sit in the audience, for about 20 seconds fists are popping out of the screen toward your face almost continuously. It is exhausting to watch, and almost impossible not to flinch.

"Alice in Wonderland." When Alice slays the Jabberwock in Tim Burton’s film version, the head tumbles down directly toward you. Try not to duck just a little. I saw this movie with my 7-year-old son, who likes to sit up front. Jonah wiggled around in his chair as if he were really there, craning to see through the crowds in the castle grounds and flinching at the looming Jabberwock. As the camera swooped and dove and the action heated up, eventually the sense of movement became too strong—he turned to me and said, “Dad, I think I’m gonna barf.” I had him close his eyes for a little bit and he was fine.

Sitting in a theater, it is not at all uncommon to experience bodily responses—to feel as if you are preparing to move, perhaps even squirming a little in your seat. You may feel as though you are ready to jump into the action at a moment’s notice, yet you are sitting perfectly still and nothing is touching you. What is going on? Your eyes and ears are telling you that something exciting is happening in front of you and your brain is preparing you to react. Of course, you know it’s just a movie. But large parts of your brain don’t process that distinction. This makes sense because our brains evolved long before movies were invented, and our perceptual systems are honed to deal with the problems posed by the real world. Our brains didn’t evolve to watch movies: Movies evolved to take advantage of the brains we have. Our tendency to want to respond physically to them highlights this.

William James described the tendency of visual images to evoke motor actions more than a hundred years ago, using the term ideomotor actions: “Wherever movement follows unhesitatingly and immediately the notion of it in the mind, we have ideo-motor action.” The term had originally been coined to describe involuntary actions during hypnosis and séances, but James pointed out that seeing something and responding automatically with an appropriate movement is one of most common ways movements are caused, “the normal process stripped of disguise.” These days, we distinguish between two different ways in which an action can be associated with an event in the world. We can describe these as two different rules that people are built to follow. I will call these the mirror rule and the success rule.

The mirror rule says “Do what you see.” Everyday life is replete with examples of the mirror rule, though we often don’t notice them. Imagine that you and I are sitting across from each other at a desk. If I cross my arms or legs, you are much more likely to cross yours. Same thing if I tug my ear or scratch my neck. As the conversation progresses, you will start adapting your accent and the pace at which you talk so that it is more like mine. At the same time, I am imitating your physical and verbal mannerisms in exactly the same way. Neither of us is likely to notice it, but the phenomenon is easy to identify in a lab. In a number of experiments over the years, psychologists have crossed their arms, scratched their ears, and varied the pacing of their speech and then measured the effects on their conversational partners. Once you start looking for them, the effects are dramatic and actually pretty funny. They also show convincingly that we mimic whether we intend to or not, often without noticing. For example, in one study an experimenter told a story about attending a crowded Christmas party, at which she had to duck to avoid bumping into someone. As she told the story, she ducked to demonstrate. A videotape of the audience showed that just as she ducked to the right, they ducked to the left, performing a mirror image of her action. In another experiment, researchers paired participants with an experimenter who either rubbed her face or shook her foot throughout the interaction. Sure enough, people who were paired with a face-rubber rubbed their own faces more often, and people paired with a foot-shaker shook their own feet. Afterward, neither group was aware that they had been performing the behavior.

There is an even more powerful way to see the mirror rule in action: Ask people to move their bodies in ways that either mirror or mismatch what someone else is doing. Suppose I ask you to perform this simple task: Stick out your hand and rest it, palm down, on the table. When a number “1” appears on the screen, raise your index finger a few inches as quickly as possible. When a “2” appears, raise your middle finger. This is an easy discrimination task, and people can do it in a few tenths of a second. Now, suppose I were to stand across from you and do the same discrimination but with the opposite movements: middle finger for 1, index finger for 2. Both of us would perform much more slowly, and if we were really trying to go fast, we would probably start making mistakes. When I do a task like this, it feels as if something outside me is making my finger want to go down just as I am trying to lift it up. It feels like it takes concentration to resist. An experiment from the University of British Columbia shows that this feeling is on target: If someone makes a startling noise while I’m watching you raise your left finger, it breaks my concentration and makes it quite likely that the finger I shouldn’t be lifting will twitch.

You can experience this feeling at the movies, too. In Stanley Kubrick’s "Dr. Strangelove," the title character (one of several played by Peter Sellers) has an involuntary habit of snapping Nazi salutes when he gets excited. You can almost feel your own arm going up. Watch "The Natural," and it’s hard not to feel yourself swinging a virtual bat along with Robert Redford. These are times when the effects of the mirror rule really stand out. I’m suggesting that this is driving your movie experience all the time, only you don’t notice it.

If you go to an action movie with a lot of kids, you can literally see the mirror rule operating. For example, during a martial arts scene, you will see a good portion of the audience waving their arms and legs along with the characters. At my house, my kids fully jump out of their chairs and leap around the living room.

More subtle, but even more ineluctable, is the mimicking of facial expression. Watch audience members’ faces sometime when you go to the movies. If a character on screen is grinning, people will tend to smile. If a character is angry, viewers’ brows will knit. If someone starts crying, mouths will turn down and you may even see tears. What is particularly striking is that this seems to work even when you don’t particularly like or identify with the character. Facial imitation is particularly powerful—and particularly difficult to override. It is part of why movies can have such a powerful emotional impact.

But for now let me mention one action that is especially subject to the mirror rule: laughing. In one study conducted at the University of Indiana in the 1960s, researchers asked people to listen to jokes either alone or in groups. The group audiences laughed more. Interestingly, they didn’t rate the jokes as any funnier; perhaps they laughed more not because they thought the jokes were any funnier, but as a direct result of seeing or hearing their fellow audience members yukking it up. The entertainment industry is definitely wise to this, whether or not producers can cite the relevant experimental studies. Sitcoms have been filmed in front of live audiences for five decades because producers know that audiences at home laugh more when they hear a live audience laughing. In fact, when a live audience does not laugh sufficiently, canned laughter will often be added to the soundtrack. And it works: People laugh more when a laugh track is added.

Why are we built to follow the mirror rule? We are an intensely social species, often dependent on other humans for our survival, and mirroring probably provides several benefits for fitting our own behavior in with that of others. Imitation is an efficient way to coordinate behavior: If you and I need to pull a rope or row a boat together, you can just say “Do what I do,” rather than trying to explain exactly how to execute and time my movements so that they will match yours. This sort of coordination is present across a wide phylogenetic swath. For example, if one of a flock of shorebirds takes off, others will tend to imitate it. They do this so that if one bird detects an approaching predator, the whole flock can quickly evacuate. Imitation is also an efficient way to learn new skills. An automatic pathway from what you see to what you do can shortcut a lot of trial and error. If you have ever studied a musical instrument or dance, you know that imitation is a powerful means of instruction.

All of the preceding examples are examples of the simple, immediate control of behavior. But in some species, ours included, the mirror rule has been leveraged to enable smarter behavior. The trick is that once a motor representation of an action is activated, you can use that representation to recognize the action, and you can use your motor experiences to make predictions about what is going to happen next. For example, suppose a man on the screen is shown grasping a doorknob. As you watch this, it activates a motor representation for doorknob grasping in your brain. Throughout your life, when you have grasped a doorknob, you have usually opened the door, and often walked through it. So, when you activate your grasping representation, this causes your opening and walking representations to activate as well. If the character then walks out the door, you have a head start on processing this sequence because your motor representation of walking is pre-activated. None of this is likely to be conscious or deliberate—it happens quickly, outside of awareness. Nonetheless, it is an important form of understanding.

Furthermore, this sort of priming may allow you access to the mental state of the character. Most of the time when you executed this sequence of movements, it was accompanied by the sense of “I am intending to walk through this door.” So observing the grasping can activate the motor sequence, which in turn can activate the intention. This chain of association can provide a mechanism for “mind reading”—for working out what characters are thinking or intending based on what they are doing. I want to emphasize that this mechanism is a bit smarter than simply imitating the behavior, but is still pretty simple—simple enough to operate fast, automatically, and outside of awareness. There is good evidence that we rely on this mechanism all the time.

In short, our behavior often follows the mirror rule, such that when we see an action performed we have a tendency to perform the same action ourselves. If the circumstances are not appropriate—say, sitting in a movie theater—we may suppress the overt execution of the action. Sometimes we’ll achieve only partial success at this. The mirror rule helps us get ready to perform appropriate actions quickly, to learn sequences of actions, and to represent the mental states of other people based on their behavior. You’ve probably heard the phrase “monkey see, monkey do” used in a pejorative way to describe shallow imitation without understanding. The mirror rule is exactly “monkey see, monkey do.” But the next time you use this phrase, think about a few things: First, it’s not just monkeys. The mirror rule is prominent in species from birds to prairie gophers to us. Second, the mirror rule is a pretty valuable trick, so don’t look on it too condescendingly. Third, although the mirror rule does indeed function without our understanding—that’s what makes it so valuable— it can provide a basis for our understanding.

Now to the second rule: the success rule. The success rule says, “Do what has worked.” It is obvious why we would be built to follow the success rule—doing what works is better than the alternative. You can feel the success rule working powerfully in situations where you need to react fast. When a traffic light turns red and your foot presses the brake seemingly before you could think of it, that’s the success rue. If you play a lot of solitaire or chess, you may notice that when you encounter a common configuration you can make the next move almost as if without thinking. That’s the success rule. The success rule is a pervasive factor in everyday action, and a significant component of how we learn new skills. It underpins a lot of “practice makes perfect.”

Psychologists talk about the success rule in terms of stimuli, which are the things the world presents to us, and responses, which are the actions we take. Its formal name is “operant conditioning.” The success rule says that if you experience a stimulus, make a particular response to it, and things work out, then next time you experience that stimulus you should be more prone to making the same response. Suppose I’m standing in a batting cage and baseballs are flying at me. I take a swing and miss low. Another swing, another miss. Then, on the third swing, I make contact. I probably cannot articulate what it was about that third swing that worked. (I may even just have gotten lucky.) But chances are my next swing will repeat a little bit of what was different about the swing that made contact. As I practice more and more, the bits that are associated with successfully hitting the ball will tend to predominate, and the bits associated with missing will die out. For most skills, particularly those we learn deliberately like batting or chess, the success rule is not the only thing operating. We may read books or get coaching from an expert, and we may consciously try to alter what we are doing. But the success rule is always working away hand in hand with these deliberate strategies to refine our performance.

The success rule shows up not just in simple motor skills, but also in more subtle, complex social interactions. Suppose you stop at the same coffee shop on the way to work each morning, and it has two lines at the counter, staffed by two regular servers. Now, suppose both provide perfectly reliable service, but one of the servers always smiles when tallying up your order, whereas the other is stone-faced. Over time we would probably see you selecting the line of the smiling server more and more often. You might not be at all aware of it, but for most of us being smiled at is rewarding and the success rule says that we will adjust our behavior to make it happen more often.

The success rule builds up habits that drive our behavior. When we go to the movies we cannot just turn these habits off. If we have experienced a stimulus repeatedly and by the success rule learned a response, we will tend to produce that response when the stimulus shows up on the screen. So, the success rule explains why you might duck a little when the Jabberwock’s head falls in Alice in Wonderland. You have a lifetime of experience with falling objects approaching our heads. A lot of them you ducked and it worked out OK. Some you failed to duck and the consequences probably weren’t good. So, when it looks like something is falling toward your head, you have been trained by your previous experiences to duck.

Toward the beginning of "The Truman Show" there is a shot in which Truman waves at the neighbors from his porch. Jim Carrey’s wave is filmed from the viewpoint of the neighbor, so it is as if Carrey is waving at you, the viewer. You may feel a tendency to wave back. That’s the success rule operating in the social domain. Ignoring someone who waves at you is rude and produces bad outcomes; waving back is friendly and produces good outcomes. Over a lifetime, this builds up a tendency to wave back when someone waves at you. This tendency follows you into the theater.

So is there a mirror rule brain center? A success rule brain center? It turns out the answer is no. So how does the brain implement the mirror rule and the success rule? To explain that, I need to tell you a bit about how the brain is organized.

The perceptual parts of our brain take up a lot of the brain’s real estate, and they are composed of a huge number of nerve cells, or neurons. Perceptual brain areas are organized according to three principles. First, many of them are neural maps of our visual world. A neural map is a representation in the brain built so that when two locations are nearby in the world, they will activate groups of neurons that are nearby in your brain. This is just like how in a road map, when two locations are nearby in the world, they will correspond with nearby spots on the road map. If you consider the map as a whole, it forms a picture of the world. How well defined are these brain maps? They vary: Some have higher resolution and some have lower resolution, and most are spatially distorted—squished. But some—particularly in the visual system—are clear enough that that they can be read off like blurry photographs.

A second principle of brain organization is that visual areas specialize, dividing up the labor of visual processing, with different visual areas focusing on different visual features. Some areas are very sensitive to relative brightness, others to color, and still others to line orientation or shape.

A third principle governing the visual brain is that areas are arranged hierarchically in successive levels. “Early” levels receive signals from the eyes after passing through only a few neurons; “later” levels are more neurons away from the retina. Importantly, each level does not just feed forward to the next level, but also feeds back to the previous level. This feedback is critical for the sophisticated processing our visual systems do.

These three principles have been studied most fully in the visual system, though we also know a fair bit about hearing and touch sensation. In vision, the earliest processing area is called primary visual cortex, or V1. V1 is located in the back of the brain, and it is where visual information first arrives when it is transmitted from the eye. Neurons in V1 are sensitive to simple contrasts of brightness, to location, and to changes over time. As we move up the hierarchy, we move forward in the brain and the features to which areas are sensitive get more complex. One area specializes in color, and damage to this area can make you color-blind. Another area specializes in motion. One set of high-level areas in the visual system is located on the bottom surface of the back of your brain. These areas receive input from upstream areas that are sensitive to visual features like color and shape, and integrate these features to play a major role in identifying and categorizing objects. These feature-sensitive visual areas feed into areas that are right in front of them, which contain cells that respond to very specific categories of objects or people—sometimes essentially to specific individuals. These cells are the closest thing we have to “grandmother cells”—cells that respond to your grandmother and only your grandmother.

As we go up each of the perceptual hierarchies, interactions across these hierarchies also increase. The highest levels in the perceptual processing hierarchies integrate information from multiple senses. Cells in these areas respond to specific kinds of objects not just when they are seen, but also when the same type of object is heard or touched. In fact, we can think of the whole perceptual system as one big hierarchy, starting from sense-specific maps that represent low-level sensory features and working up to cross-modal maps representing abstract features of our environment.

The motor system is also organized hierarchically, but the motor hierarchy can be thought of as the reverse of the perceptual hierarchy. Whereas the earliest levels in the visual hierarchy are closest to the eye, the latest levels in the motor hierarchy are closest to the hand. The last stage before most voluntary motor commands leave the brain is primary motor cortex. Primary motor cortex occupies a strip from along the middle of your brain, running from the crown of your head down to just in front of your ears. It controls simple movements of your body, and it is organized as a map of your body with the feet represented on top and the hands toward the bottom. Higher levels in the motor system control more complex movements.

The perceptual hierarchy and the motor hierarchy are both changing and adapting all the time. Specifically, the connections between neurons are adjusting to increase or decrease the effect that one neuron has on others. Neuroscientists call this plasticity. Plasticity is critical for adapting behavior to a changing environment; it is our brain adjusting itself, tuning itself up over time to work better. Plasticity is responsible for much of the operation of the mirror rule and the success rule. Let’s start by looking more closely at the mirror rule.

In "The Karate Kid" (the original, from 1984), Mr. Miyagi gives Daniel the task of waxing a car. He shows him the proper technique—“wax on” (a circular motion), “wax off” (a circle in the other direction). Daniel executes probably a thousand repetitions of this action before he is done. On each repetition, he is performing a motor action and experiencing the perceptual consequences of that action. This gives plasticity a chance to adjust connections between his neurons to strengthen the association between what the movement feels like and what it looks like (and sounds like and feels like). It is no surprise that when Daniel needs to produce the same movement for karate, now without the waxing cloth, he is smooth and proficient.

Most of the actions you mirror are not subject to this kind of intense practice, but it helps us to see what is going on. In everyday life you have vast amounts of experience acting and experiencing the perceptual consequences of those actions. Plasticity leverages that experience to tune up the connections between your perceptual system and your motor system so that you can imitate a vast repertoire of movements. Importantly, you can stitch these bits together in novel ways, like assembling a new sentence from your preexisting vocabulary. Often we find ourselves imitating new combinations of actions that we may not have experienced before. This generativity is just what makes the mirror rule so useful.

Excerpted from "Flicker: Your Brain On Movies" by Jeffrey Zacks. Published by Oxford University Press. Copyright © 2014. Reprinted with permission of the publisher. All rights reserved.


By Jeffrey Zacks

MORE FROM Jeffrey Zacks