The quest for the perfect game face

Video game designers are racing to create characters that feel real. Now, if they could only turn digital figures into flesh and blood.

By Matt Shaer

Read more: Video Games, Technology & Business, Tiger Woods, Computers, Arts & Entertainment, Arts & Entertainment Features, grand theft auto

Feb. 19, 2008 | Early in his career, David Cage made the decision to discard what he calls the "traditional mechanics" of game design -- a puzzle, a solution, and a chest of gold.

Cage, who is fond of saying he is in the business of "creating emotion," is best known as a video game designer. A few years ago, his studio, Quantic Dream, released its first major title, "Indigo Prophecy"; it went on to sell more than 700,000 copies.

The success, Cage recently explained, was spurred by the game's open-endedness: in "Indigo," plotlines spill, amorphously, in several directions, none exactly wrong and none exactly right. Audiences are forced to view each level dynamically. But shortly after "Indigo" was launched, Cage began to receive complaints about the game's aesthetic realism, which one reviewer labeled, disparagingly, "atmospheric, but not stellar." The most significant flaw was the face of Lucas Kane, the hero of "Indigo Prophecy." In some scenes, Kane's face looked wooden; in others, the muscles around his mouth moved too much, giving him an eerie, reptilian quality.

"Human emotions are expressed pretty much by every single part of our body," Cage says. "Think also about what emotions can be expressed with hands. Your skin says as much -- you can sweat if you are nervous, be blank if you are scared, blush if you are confused, etc. "None of this is spectacular in itself, but all these little signs contribute to the expression of emotion, and we are used to decoding them unconsciously."

When a video game designer fails to accurately encode those expressions, the results can be disastrous. Consider "Grand Theft Auto IV," the latest installment of the bestselling franchise. In the game's trailer, the hero, Niko, speeds across a gritty cityscape, waving a chrome pistol out the window of his car. He shouts at pimps, whispers to gangsters, and sprints down a dark alley. Later, he pilots a speedboat down a crowded canal, while gunfire rains overhead.

For the most part, the rendering in the trailer is superb: the water ripples, the mist floats, the muzzle flashes are bright and sharp. Niko, meanwhile, is imbued with a robotic stoicism. His eyebrows are thick and unresponsive, and when he frowns, his forehead turns rubbery, like a twisted plastic bag.

"A face is extremely complex in its muscular structure, and if anything is not perfectly right, you will immediately notice," Cage explains. A still life, in other words, is much easier to create than a portrait. We look at faces every day; we have taught ourselves to look for signs of sadness, or happiness, or deceit. It is a survival mechanism. If a digitalized face appears "broken" or rubbery, it destroys our sense of the illusion.

If a developer could create a photorealistic face -- and make it move, and respond like the flesh-and-blood original -- gamers would become doubly immersed in the game. So a few months after the release of "Indigo Dream," Cage set the team at Quantic to work. Their first step was to experiment with a combination of synthetic animation and motion-capture technology, which uses the movements of an actor.

From there, producers developed specific rendering systems to produce, in Cage's formulation, "a skin that does not look synthetic but has enough details to look real"; "reflections in the eye that give the feeling that the eyes are wet"; and a clammy moistness in the character's brow.

As a test, Cage wrote a teleplay for a short film, "The Casting." The plot was simple: Mary Smith, a 24-year-old actress, is trying out for a part in an upcoming project. She is nervous, she explains to the casting agent, who sits somewhere out of view -- there have been other auditions, and even a few big parts, but "there's always something wrong with me."

As Smith begins to read her lines, she grows more confident and slips quietly into character. Her lover is cheating on her, Smith says. She knows this because she followed him to a hotel where she met the girl: "Then my whole world falls apart." As she speaks, Smith quietly produces a pistol and presses it to her temple. She doesn't shoot; the bullet, she explains with a fierce intensity, is intended for someone else.

"The Casting" debuted at the E3 gaming summit in Los Angeles, on May 16, 2006. It was greeted, instantly, as a resounding success -- an example of what could be done with digital technology. Still, what Quantic Dream had produced was not a game. There was no interactivity to speak of, except the Play button. For months after the debut, Cage and his team remained relatively quiet about how they intended to use the technology, and the industry chatter subsided.

Then, in December 2007, in a widely read interview, Quantic executive Guillaume de Fondaumière announced that the studio had signed an exclusive deal with Sony to develop a game for the PlayStation 3. The game, he said, was called "Heavy Rain," and it would feature "hundreds" of realistic characters, which would each be rendered with the same technology used to "create" Mary Smith.

De Fondaumière added that the "uncanny" effect that had hobbled scores of otherwise perfect games was a relic of the past. Quantic's new system had been maximized to pick up "very, very small motions, which means we can not only capture full body movement, but also facial movement and expressions."

Undistracted by rubbery digital faces, gamers would finally experience total emotional immersion.

---

The idea that a gap in realism triggers mental unease was first conceived by a German psychologist named Ernst Jentsch. In 1906, Jentsch published "On the Psychology of the Uncanny," where he theorized that cognition is divided between two poles: the "new/foreign/hostile" and the "old/known/familiar." If the brain can not immediately associate an object or experience with one or the other, Jentsch wrote, it becomes stuck somewhere in between, mired in doubt that "only makes itself felt obscurely in one's consciousness."

Imagine, for instance, that you are standing in the middle of a dimly lit room, surrounded by wax manikins. As Jentsch notes, your imagination will probably begin to swell. Is one of those dolls actually a human? What if it begins to move? Will I scream? (Ridley Scott plied a similar fear to great effect in "Blade Runner.") Jentsch suggested that anxiety is "repeatedly and automatically aroused anew when one looks again and perceives finer details."

Jentsch's theory lay essentially dormant for the next half century (it is referenced, tangentially, in a series of essays by Freud) until 1970, when Masahiro Mori penned the influential essay "The Uncanny Valley." Mori, a Japanese roboticist, was fascinated by the emotional and psychic reactions humans have to "humanoids." Famously, Mori noted that "human beings themselves lie at the final goal of robotics," but the closer roboticists came to simulating a realistic face, the more noticeably a human would recoil from its appearance.

For instance, what if a designer tacks a plastic mask to the front of a robot? That increases the "familiarity," and a viewer might begin to anthropomorphize its movements. This humanizing effect is subtle, and reassuring; one is cognizant that the machine is still a machine. Conversely, when that same viewer is presented with an aesthetically accurate "humanoid," the prevailing mood often turns to dread.

Next page: Game designers struggle to render the simple movement of a human jaw or a wrinkled brow