Slaves in the family

A white man's odyssey into his family's secret history is a searing look at the most shameful event in America's past.

Published February 27, 1998 7:02PM (EST)

There is a saying among Southern blacks that goes like this: White folks up North don't care how high you get as long as you don't get too close; white folks down South don't care how close you get as long as you don't get too high. In "Slaves in the Family," Edward Ball reveals just how intimate the relationship between Southern whites and blacks can get. In this epic family history, he shows how the "peculiar institution" of the past has linked the progeny of slaves and slave holders through time and blood and memory.

Ball is descended from a dynasty of Charleston, S.C., rice planters who, from 1698 until the end of the Civil War, owned thousands of African- and American-born slaves. Although the family's great fortune ended with the abolition of slavery, the legacy of plantation days lingered in each generation. Ball, son of an Episcopalian minister and great-grandson of a character known as Isaac the Confederate, grew up hearing romantic, vaguely troubling stories of his antebellum ancestors.

"There are five things we don't talk about in the Ball family," Ball's father used to joke. "Religion, sex, death, money and the Negroes." But Ball knew this was not entirely a jest. Before he died, the minister gave his young son a leather-bound copy of the Ball family history, written at the turn of the century by a cousin. "One day you'll want to know all about this," he told him.

Years went by. Ball graduated from an Ivy League college and moved to New York City. He worked as a freelance art critic and wrote a column about architecture for the Village Voice. The family history book remained on his shelf. Still, he writes, "The plantation past was etched in my unconscious." It was finally jarred to the surface with the arrival of an invitation to a Ball family reunion in the summer of 1993. Ball decided he would return to Charleston, to "face the plantations" and find out what they meant to him.

The author joined 150 relatives for a chartered cruise up the muddy Cooper River. It was here the Ball family once owned thousands of acres of rice fields and grand estates with names like Comingtee and Kensington and Halidon Hill. All that remained were swamps and tract housing and a few festering ruins. Ball turned his attention to the living relics around him, descendants of the vanquished gentry. He found them a diverse bunch: "Some of the family had manners, others none; some had money and status, some neither," he writes. "But inwardly the plantations lived on."

While Ball knew a great deal about the history of his white forebears, he knew almost nothing about the slaves who lived among them for six generations. Their presence, he recalls, was "like a puff of black smoke on the wrinkled horizon of the past." He decided to find out who they were and what became of them, "to make the story whole."

The writer moved into a crumbling ancestral house in Charleston. Armed with the old family history, slave records and library archives, he started to dig into the past. Early in his effort, he was warned by an older relative, "To do this is to condemn your ancestors! You're going to dig up my grandfather and hang him!" The warning was prophetic.

Ball is not a neutral narrator. A preacher's voice sometimes rings in his evocative, beautifully crafted prose. His message is that slavery was an abomination and no one who benefited from it -- directly or indirectly -- is without the stain of guilt. To those who would protest -- "I never owned slaves, what's it got to do with me?" is another refrain you hear down South -- Ball provides a simple response: To live with the advantages of white skin in America is to benefit from the old slave system. At the same time, he knows that what's past is past.

"Rather than feel responsible," Ball writes, "I felt accountable for what had happened, called on to try to explain it." In answering that call, Ball is seeking a form of redemption -- and his writing becomes an act of contrition.

The story of the Ball plantations begins in the 17th century, with the arrival of the author's English ancestors in America. In 1698, a semiliterate 22-year-old farm boy named Elias Ball inherited half of the Comingtee plantation from a distant relative and booked passage to the colony. This founding patriarch, known in future generations as Red Cap because of the headgear he wore to cover his baldness, found himself master of 20 or so African and Native American slaves. He expanded the family business in every way. Old Red Cap acquired new lands and sired five white children who lived to adulthood. He also, the author discovered, had a relationship with his house slave, Dolly, that resulted in at least two mulatto offspring. The white children inherited a vast fortune. His black son, Edward, although eventually freed, lived his life in hard labor in the Balls' stables, tanning hides and tending horses. (The other child died young.) Not unexpectedly, in the course of his research, Ball found enough miscegenation to fill a Faulkner novel.

- - - - - - - - - - - - - - - - - - - - -

The author said in an interview with the Charleston newspaper that he never intended to disgrace his family in writing this book. But there is no way to read the history of the plantations without revulsion. Ball contrasts the indolent splendor of his early ancestors' lifestyles with the brutal subsistence of the slaves and their descendants. And he debunks the fable with which the latter Balls comforted themselves: that they were gentle masters who cared for a simple people. The slave owners he exposes were neither gentle nor paternalistic; they bought and sold humans for profit whenever it suited them. The family archives brim with instances of cruelty: The toes of runaways were chopped off, malingerers were flogged. In a letter to her husband, Ann Ball, mistress of several plantations in the early 1800's, airily describes beating a slave called Betty with "the little whip in our dressing chamber" for washing towels "badly." The beating apparently didn't cure Betty of her "impertinence," because she was then sent to the dreaded Work House in Charleston, where slaves were punished with hard labor and torture.

Ball finds in plantation records enough evidence to put to rest another myth: that slave families were never separated. He describes an auction in 1819 in which a large Ball estate was liquidated and dozens of slave families were divided and sold off. Later, Hugh Swinton Ball, a parasitic great-grandson of Red Cap, would often sell a few slaves at a time when he needed cash for gambling or travel.

In rewriting the family legend, Ball crushes some of his own cherished memories. In his research he learns that the placid dunes of Sullivan's Island, where he spent part of his childhood, were the site of the Pest House, where slaves were held in quarantine after their voyage. Their bodies were buried in mass graves. He writes, "My childhood idyll has dropped its mask, and I sometimes shovel the graves in my sleep."

There is no easy way to categorize "Slaves in the Family." It's part "Absalom, Absalom," part "Roots," with more than a dash of "Gone With the Wind" thrown in for good measure. African-American genealogy has been a robust genre since Alex Haley published his bestseller a generation ago, and accounts of white Southern heritage are common as ticks. The legacy of slavery, the bitter romance of the Lost Cause and the guilty thrill of miscegenation have long been staples in Southern literature. But Ball's commingled history of his ancestors and their slaves has the feel of something different. It is a patchwork of vignettes and snippets and historical accounts, interspersed with the author's sometimes comical, sometimes heartbreaking encounters with his contemporary white and black kin. For example, in modern Charleston (which may be an oxymoron), Ball introduces us to a venerable white social club where members sip bourbon and perform Negro spirituals from the old plantation days in full Gullah dialect. "I think the slaves couldn't have done it better themselves," Ball shakily assures them.

Ball draws mixed reactions as he tracks the descendants of his family's slaves. He is met with wry amusement by Edwina Harleston Whitlock, a woman in her 70s with sepia-colored skin, who greets the author with the words, "It's a great pleasure to meet a long-lost cousin." They proceed to sit down and match relations, just like any new acquaintances would do in the family-obsessed South. Through this encounter, we learn about the amazing Harleston family of Charleston, scions of a union between a Ball cousin and a slave woman, which produced generations of artists, musicians and intellectuals. (This leads one to believe that the big Ball secret was not that there were blacks in the family, but that the blacks were more interesting and talented.)

Another meeting with a distant black relative called Leon Smalls is less congenial. Smalls, a middle-aged union representative, vents his anger at the white man at his door. The light skin and hair he inherited from the Ball family had only caused his schoolmates to taunt him. A life of insults and slights had made him a bitter man.

No one is safe from Ball's quest for accountability. He even travels to Sierra Leone to confront the progeny of the African chiefs who sold their own people to the English slavers. Some of their defensive comments sound a lot like those of the white bankers and golfers of Charleston. "We should try to forget, and live for today," says one professor's wife. In the end, the child of the slave owners and the children of the slave sellers find a way to acknowledge the original sin that connects them.

At the heart of "Slaves in the Family" is Ball's encounter with Emily Frayer, an ancient black woman with a saber-sharp memory. Her grandmother, Elsie, was a former slave who had told her stories about the Ball plantations. Frayer recites them as if she were living them herself. She tells him what the slaves wore, when they ate their meals and how they buried their dead at night because they had to work in the fields all day. And she tells how her grandmother dropped to her knees in joy at the end of the Civil War when Sherman's troops rode up to the big house and told her she was "free as a bird."

Ball takes Frayer back to the ruins of the cabin where she was born at Hyde Park, the last plantation to pass out of his family. It is here amid the weeds and rubble under the cool shade of pines that Ball offers his apology.

"I'm sorry for what my family did to your family," he says.

She replies, "It wasn't in this time at all, so you must don't feel bad. You could have come a long time ago, but you come in due time."


By Maryanne Vollers

Maryanne Vollers is the author of "Ghosts of Mississippi." She lives in Montana.

MORE FROM Maryanne Vollers


Related Topics ------------------------------------------