But the Aniston we see in “The Break-Up” is a woman who has plenty to prove — perhaps too much for any actress. In some of her earlier roles, her line readings came off as both sharp and light at once, as if she had nothing to lose. But in “The Break-Up” — in a role that is, admittedly, not particularly well-conceived — all we see is a woman who’s desperate to hang on to what she’s got. Aniston plays Brooke, a wound-tight art gallery employee who gives off conflicting, unreadable vibes: Even in her trim black sheath dresses, she’s like a sullen cheerleader. (Her boss is played by the marvelously crisp Judy Davis, in a Louise Brooks bob that looks as if it’s been cut from a sheet of glass.) Brooke has been living with her lovable lummox of a boyfriend, Gary (Vince Vaughn), for several years. The two host a disastrous dinner in which they try to get their families to socialize — Brooke’s mom is played by Ann-Margret, who has too much sparkle to have believably spawned such a wet blanket — and, afterward, the stressed-out Gary refuses to help clean up.

As happens so often in real life, this relatively small matter is a spider crack that rapidly widens into a continent-dividing fissure. Brooke may be high-strung, but she is the motor that keeps the household working smoothly; Gary is marvelously easygoing, always ready to make her laugh (and there’s one allusion to his generosity in bed), but he’s as lazy as hell and doesn’t come close to pulling his weight around the condo.

As the two fight, their mutual bitterness expands and explodes into something that can’t be shoved back into the bottle — a demon has sprung to life between them, a fiend of their own creation that’s so powerful they can’t vanquish it, together or separately. The scene is wrenching to watch — Reed orchestrates it so that it plucks all too effectively at our nerves, and both actors rise to the occasion: Aniston, her mouth set in a tight little line, is both deeply unlikable and painfully sympathetic; Vaughn, at first acting as if he has no idea what she’s complaining about (a perfect indicator of the way he sloughs everything off too casually), finally lets loose with every resentment he’s kept coiled tightly inside — they fly out like snakes from a joke peanut can, and they’re anything but funny.

Before we know it — almost before they know it — Brooke and Gary have broken up, and right about then, like their doomed love, the movie around them also begins to fizzle away. It’s perfectly possible to make a good comedy that doesn’t make you laugh out loud, and that may be the effect Reed (who directed the spiky-smart “Bring It On”) and screenwriters Jeremy Garelick and Jay Lavender were going for.

But to do that, you need to be able to write little jokes that hook into you like parasites, and “The Break-Up” doesn’t have many of those. The gimmick here is that Gary and Brooke continue to live in the same space, even after they’ve decided to call it quits. Gary, passive-aggressively, puts a pool table in the dining room, as he’s always wanted. Brooke apparently (it’s never really explained) allows her brother (the wonderful John Michael Higgins, from “A Mighty Wind”) and his a cappella group to practice in her bedroom. These tepid little gags shuffle in, one after the other, and none of them really build on the excruciating vitality of the movie’s early (and best) scenes. “The Break-Up” doesn’t know whether it wants to be a facile, enjoyable date movie or an unnerving examination of the dark, pockmarked underbelly of everything we expect out of romantic relationships, and it settles for a deeply unsatisfying nowheresville. Similarly, the movie’s ending hovers in a tentative, vague universe. After test audiences complained about the original ending, Reed reshot it several different ways: The one he went with is admirable in some respects (it certainly isn’t a sop to the expectations of the audience). But it doesn’t finish the picture off in quite the way it needs, either — there’s a difference between leaving your narrative purposely ragged and simply cutting it off at the knees.

A comedy of remarriage — or, possibly, not — like “The Break-Up” obviously rests on the strength of its two leads. But almost every actor here (including the couple’s real estate agent, played by Jason Bateman) is more interesting than Aniston. Vaughn, in particular, works hard to give the movie some decent grounding. His timing is always fabulous, and here, his jokes sometimes inch the movie into some potentially uncomfortable territory on tiny needle feet. Vaughn is fascinating to watch: He’s always alive, sometimes alarmingly so. He delivers his lines as if he’s a guy who might blow at any minute; when the camera catches the whites of his eyes at just the right angle, he looks something like a bull that’s just about to be let into the ring.

But is this a killer bull, or are we talking about Ferdinand? Vaughn doesn’t make the mistake of playing anger instead of playing a character. In his best scenes here, he shows us how Gary’s snappish jokes are the flimsy veil he holds up to guard his feelings — a kind of invincibility cloak.

Aniston’s Brooke has her defense mechanisms too, of course. (The chemistry between Aniston and Vaughn, who are ostensibly involved in a real-life romance, feels only businesslike here.) But even though “The Break-Up” is clearly laid out to make us see both sides, Brooke is much harder to warm up to — and that would be OK if only the movie, or Aniston, recognized it. Instead, Aniston’s Brooke sticks cautiously to cutie-pie cuddle-bear territory, even though we can see that she really has a core of steel. I don’t think Aniston can reconcile those conflicting qualities, not in the character of Brooke, and possibly not even in herself.

The astonishingly gifted Goldie Hawn had similar difficulties: She so craved the love of her audience that she learned to paint a dewy glow around the edges of her performances, instead of revealing them to us in all their fierce and glorious sharpness as she did in pictures like “Shampoo” and in the (unfortunately, nearly impossible to see) director’s cut of Jonathan Demme’s “Swing Shift.” Aniston isn’t as inherently talented as Hawn is. Still, when I look at her — with her cute little doll’s nose, and those my-best-friend’s-girl greenish eyes — I want not just to like her, but to find her interesting. It’s possible, of course, that her only gift is to hint that there might be something going on underneath that adorable surface, when really there isn’t much. At best, she may simply be a master of shallow illusion — not the least of which is the tantalizing promise that maybe someday she’ll give us a performance we can’t turn way from.