The haunting of Carly Simon: "I don’t feel that I’m living in James’s house. But there are constant memories"

Salon talks to the singer-songwriter about her bestselling memoir, her parents, her ex James Taylor and forgiveness

Published December 25, 2015 9:00PM (EST)

  (Heidi Wild)
(Heidi Wild)

Like most artists, Carly Simon is haunted. Throughout her storied, nearly five-decades-long career as one of our finest singer-songwriters, she has often grappled with the specters of memory, family and relationships in songs like “That’s the Way I’ve Always Heard It Should Be,” You’re So Vain,” “It Happens Everyday,” and even a more obscure tune titled “Haunting.” Backed by a rising cascade of piano, oboe, harps, drums and a gothic choir, Simon sings—both as gently and as fiercely as a ghost itself—“There’s always someone haunting someone haunting someone.”

Such piercing poetry, candor and vulnerability have long been the trademarks of Simon’s music, and she brings these qualities to the page in her latest venture, the New York Times bestselling memoir "Boys in the Trees," published last month by Flatiron Books, along with an audiobook read by Simon (with an original score she wrote with composer Teese Gohl) and a companion album titled "Songs from the Trees." A riveting work that stands alongside Patti Smith’s "Just Kids" and Rosanne Cash’s "Composed" as one of the finest music memoirs of the past few years, "Boys in the Trees" chronicles the ghosts that have often acted as both muses and torments for Simon in her creative and personal lives.

As the third daughter of four children born to the self-made publishing magnate Richard L. Simon, co-founder of Simon & Schuster, and his wife Andrea, she grew up in comfortable homes in New York City, Connecticut and on Martha’s Vineyard, where her parents’ dinner guests often included the likes of George Gershwin, Benny Goodman and Jackie Robinson. But the façade of a sparkling, creative family papered over tumultuous dynamics at play. In shattering prose, Simon describes her “inability to get and keep Daddy’s attention, and the suspicion that of his four children, I was the one he cared for least.” She recounts feeling inferior to her talented older sisters, of a disturbing early sexual experience with a teenager in the neighborhood, and of her mother’s affair with Ronny—a young man who had been hired as a babysitter for Simon’s younger brother—that created lingering repercussions for the entire family.

Simon traces the legacy of those events as she recalls her early musical career—first as one of The Simon Sisters, a promising folk duo with her sister Lucy, and then as a reluctant solo artist climbing the charts with her own confessional hits—and romances with some of the most handsome and creative men of the day, including Cat Stevens, Kris Kristofferson and Warren Beatty. But much of the memoir is reserved for the love that made the most lasting mark: her marriage to James Taylor, which ended in 1983 following a decade of passion and turbulence that fueled each other’s music and produced two children, Sally and Ben, both singer-songwriters themselves.

Throughout the memoir, which concludes with her divorce from Taylor, Simon returns to the story of Orpheus—the mythical musician who attempts to rescue his departed love from the clutches of the Underworld—and applies the shifting meanings of that epic tale to her own life. Although Orpheus was unsuccessful in his quest, Simon triumphs in rescuing her own younger self in a book that haunts the reader “like an echo the wind leaves, unreconciled.”

I spoke to Simon last week via telephone from her home on Martha’s Vineyard as she prepared for her holiday celebrations. The interview has been edited for length and clarity.

"Boys in the Trees" has made it to number 7 on the New York Times Bestsellers List in Nonfiction. How does that feel, for such an intimate project you’ve worked on for so long, to be finally out there and meeting acclaim?

Amazing. I’m still trying to feel how it feels...When I wrote it, I had really no belief that it was going to ever be published, and so that’s why it has such an intimate feel, that’s why it has such a close-to-the-bone sensibility. And then when...there was a decision made that it was going to get published pretty much as-is, I didn’t have time to look back and say, oh, well that’s too personal. So I just went with it. And I just thought, well, the best gift that we human beings have to share is the gift of ourselves and what we have to offer. But it doesn’t make any sense if it’s not honest, if it’s not the real deal. I think a lot of what people are responding to is just the honesty of it.

Did you find that prose came naturally to you?

Yes. I had kept diaries for years, which is why I have such detail in the book. I mean, it’s lifted pretty much from my diary—not word for word necessarily, even though sometimes it is—but the publisher decided that he wanted me to paraphrase that which was written in diary form to make it less dear diary and more just the feeling and the event right on the page as it happened. So sometimes I feel like I’m lifting from myself. I guess that’s allowed.

That’s folk music.

That’s folk music. That’s right. [laughs]

In the 1980s, your late friend Jacqueline Onassis, who was an editor at Doubleday at the time, contacted you to write your autobiography. You started [writing] but eventually abandoned the project and instead published a number of successful children’s books. Why weren’t you able to complete the [book] then?

I only wrote about 80 pages before I realized that the nucleus of the story was going to have to be altered in order to fit the emotions of the time. In other words, I just didn’t feel ready to do it. Now, what I eventually did is not that much different—the only thing is that this time I really didn’t have a choice, because it was now or never, kiddo. But also my mother was alive, and there was a lot that my mother obviously would have been very uncomfortable with. I hope in her current resting place she is not too uncomfortable with it, but—that made a big difference to me at that point. I didn’t want to embrace the material the way I’m embracing it now.

But you found that earlier project did influence this one.

It only influenced me inasmuch as [Onassis] thought that I had an interesting story to tell. She didn’t even know me at the time—this was the first time that we actually made acquaintance...I mean, can you imagine? I got a call in the middle of the afternoon saying, [imitates Onassis’s distinctive voice] Carly, this is Jacqueline Onassis. And of course I didn’t believe her...But her voice was—it was either her or a very good sound-alike, and so I didn’t make a fool of myself, as I have in the past when I haven’t believed that somebody has been on the phone and it was really them.

In talking about "Boys in the Trees," you make a clear distinction between autobiography and memoir, which I think is an important one. How would you describe that difference, and why did you want to write a memoir instead of an autobiography?

[In] a can pretty much skip around in time. You don’t have to stick to a chronological order. In the end I did end up kind of chronologically, but [with] memoir there’s much more poetry allowed. An autobiography is...a biography of you, written by you...It still relies on chronology and facts and order, and a memoir is more free-form. It was very important to me, that distinction, because I very much wanted to not have to stick to a guideline that was time-dependent.

In the first section, the one devoted to your childhood, you write, “The biggest secret and vanity of the Simon family was to insist that nothing was wrong when, in fact, so much was wrong, and neither one of my parents ever owned up to it.” Those are powerful words. Your father died when you were a teenager, but as an adult, were you ever able to have a conversation with your mother in which you directly addressed that?

I actually did, and she was still in a state of denial about it. She was still with excuses about why she did this and that, but she never really owned up to the fact that she broke a lot of our hearts when we were very young and vulnerable, as well as my father’s heart, by her decision to allow Ronny into the house and live there for so many years under our gaze and still really denying it the whole time, as if we were seeing something and we were not allowed to know that we were seeing it. We were told that what we saw was a mirage. And that’s a very hard thing to swallow.

I think my older sisters...had an easier time with it. My sister Lucy was kind of caught in between, because she was often asked to take my father on trips so as to leave my mother and Ronny alone. But then I was left in the house with my mother and Ronny and my brother, and my elder sister was off at school. And Lucy—who was my father’s favorite and whom my father depended be his comfort—was really caught in the middle because she wanted to please mother and she knew about the predicament that mother was in. But she didn’t ask as many questions as I would have liked to. I was the one that rebelled...I took to punching who I saw as the interloper, and I just said, Get out of this house, get out of this house, all the time as an 8-year-old. And I had friends come over from school and [try to] beat him up. [laughs] We were all 9-year-old little girls trying to beat up this big man.

That’s so much pressure and responsibility for young girls to deal with.

Yeah, there was much too much responsibility that was left to us, and I recognize that my mother was in a terrible position herself, and she was miserable in her marriage and she didn’t want to be there, and she felt that her husband wasn’t treating her well...

The book originally began with a chapter...where I drove up to see this 92-year-old woman who had been an amore of my father’s while my mother was pregnant with me. And this was always the excuse that my mother used for allowing her lover not only in her heart but into the house. I went up to see [the woman] and asked her whether it was true or not. And that will have to come out later. It wasn’t in the book. It didn’t get in the book because the publishers didn’t feel it was a good beginning.

One of the themes of the book is of unresolved relationships—with both your parents, of course, and then with your former husband James Taylor. At this point in your life, are you at peace with all of those relationships?

Well, I would say that the relationship is very resolved from James’s point of view, which makes it clearly impossible for me to have a relationship with him because he’s put down an iron curtain, which I think is a very strange thing to do since I never considered myself to be harmful or dangerous, but perhaps he knows something that I don’t know. But it’s been very hard to raise children without having their father to talk to. It’s not as if he’s dead somewhere—he’s living in the Berkshires. So it’s been much more preoccupying to me than if he had been able to be reached. Then it would have been normalized. But being an abnormal situation, one tends to rebel against it.

Do you ever see that changing at this point?

Well, if you ever see him [laughs], say, what the hell are you doing? I mean, it certainly is very noticeable. It’s highly noticeable to everyone around, and it puts people at an...unrealistic disadvantage because they feel as if they have to take sides still, which is just an absurd notion—

Thirty years later.

More than that, yeah. So I don’t know how it would [change] because it’s so public now, his disavowal of me...When I wanted to sing at Boston Strong [a concert held in the aftermath of the 2013 Boston Marathon bombings], I was told that I couldn’t, and the reason was that he was going to be there. So there are all kinds of not only political but musical events that I’m just not allowed to be a part of because everybody knows James won’t do it if Carly’s there.

In spite of all that, I was so struck in reading the book by the generosity with which you write about him. You portray him with tenderness, but you also don’t shy away from confronting the complexities of his character. How did you manage that balancing act?

It’s interesting, because I am very—I love him. I love him dearly. But I love him in a vacuum. You know, I love him in the way you would love a god, but maybe a slightly malevolent god. But it is unconditional love, and...I think when I married him and took those vows, I was deeply, deeply serious, and even though our relationship didn’t meet all the standards of the vows that we took, the I will love you forever vow certainly stuck. And I wouldn’t be surprised if somewhere there was a complicated feeling in his soul—that that isn’t true of him as well, but it’s too dangerous for him to feel.

But I didn’t have a hard time observing him from a very positively subjective point of view—which includes the objective, too. I do love him, and I think he’s fascinating and brilliantly talented. And after all, how could I not love him, because he’s the father of my children. And I just don’t see how that’s possible. I guess some people can do it, but they have a better ability than I do to turn all my feelings off.

The house that you built together on the Vineyard is an ever-present character throughout the book, and of course you still live there. Is it hard to live in a place in which so much poignant history is always present, or at this point is there a sweetness to that?

Oh, it’s very sweet. I mean, I look at all the strange angles [of the house]. The first parts of the house were all planned by James. And there are these wonderful dormers, very high and pointy, and it’s hard to put pictures anywhere because all the walls are slanted and very cathedral-like. And of course, everybody tells me, The house is yours now—and I do feel that. I don’t feel that I’m living in James’s house. But there are constant memories,’s not that we moved into some place together. We built it together...For the first five years or so [after the divorce] I kept on saying, And this is James’s, this is where he kept his tools—I would take people around as if it was James’s museum. And I don’t do that so much anymore. But even when I did it didn’t seem odd. It just seemed like, that’s the way it is, just the way I would do it if I was living in the house that I was born in. I would say, well, this is my father’s study and this was where my mother used to make curtains and this is where my uncle led the Civil War from. [laughs]

Throughout the book you return to the story of Orpheus, in your words, “a mythical god [who] could somehow deliver me from darkness.” I’m wondering where you find Orpheus in your life today?

I think I was always trying to get to know Orpheus, and now I realize that Orpheus is music, and it’s the way that music summons me, and it summons me out of pain all the time—it’s the one thing that can constantly do it, it’s listening to music or making music.

It reminds me of that scene earlier in the book right after your father has died, and [you and] your sisters sing “The Lord Bless You and Keep You.”

Yeah. That was the one in the kitchen where we were having Rice Krispies and singing. The three sisters, we really always sang together...and we always made up three-part harmonies. Our voices were particularly suited for the three parts, because my middle sister Lucy is a soprano, and my elder sister is a mezzo-soprano, and I’m like a tenor—even as a child I was kind of like a baritone-tenor. I always had a very low voice.

I heard you say [recently] that you’re starting to lose a bit of your voice, some of the high notes. How is that affecting you, and are you still planning to record?

Yes! I want to use what I have, and I want to use it to its best advantage. And it just means that I’ll be singing probably in a lower register—you know, just more of those smoky tones will come out. But it doesn’t limit me. I mean, if I’ve lost a note at the top I’ve gained one at the bottom.

It’s the holidays and your album "Christmas is Almost Here" is a perennial favorite of mine, and one song that I always keep coming back to is “Forgive,” which is prayer of sorts [that says] “Help me to make the loving choice.” Has writing this book helped you to reach that goal?

That’s very nice of you to bring that up, because...whenever I was tempted to lash out and present a less generous side of my soul, I pulled back and made a decision to make the loving choice. And, you know, sometimes I asked for that help...At times—certainly during the course of the three years [of writing the book]—there were incidences that happened where my hurt seemed very fresh. And when my hurt is fresh I can’t [have] a well-balanced attitude. And so there were temptations to lash out, or to put things in ways that...would have only been true of the moment and not an overall picture.

I’d like to bring it back to your father, because so much of the book—and I think so many of the themes of your body of work—refers to him, to your relationship with him. You write very movingly about him and how you never felt pretty enough, talented enough, to gain his attention as a child. But with all that you’ve accomplished today, how do you think he would respond? Do you think he would be proud of you?

I think once the relationship is established when you’re that young that there’s a temptation to always make the facts fit the feelings, and I think that I’ve gone through my life looking for men who were similar to my father who would continue to make me feel not as good about myself as I might have if I’d had a different father or if my father had felt differently about me...Father-love is so important, and I think early on it’s extremely important because those patterns are set then. So I think that there were moments that I enjoyed a sense of being really seen by him, but not in the same way, for instance, I feel seen by my children. But it’s been hard for me in my relationship with men because there’s always that temptation, as I say, to continue the pattern. I have to be aware of it all the time and not let that happen.

By Jason Kyle Howard

Jason Kyle Howard is the author of "A Few Honest Words: The Kentucky Roots of Popular Music" and coauthor of "Something's Rising: Appalachians Fighting Mountaintop Removal." His work has appeared in the New York Times, The Atlantic, Oxford American, Salon, The Nation, The Millions, Utne Reader, and on NPR. He directs the creative writing program at Berea College and serves on the faculty of Spalding University's Naslund-Mann Graduate School of Writing.

MORE FROM Jason Kyle Howard

Related Topics ------------------------------------------

Books Boys In The Trees Carly Simon Editor's Picks James Taylor Memoir Music