Are taxes racist? Author Dorothy Brown on how the tax code makes the wealth gap worse

Author of "The Whiteness of Wealth" explains how the tax code privileges whites — and punishes Black people

Published May 5, 2021 6:00AM (EDT)

The Whiteness Of Wealth by Dorothy A. Brown (Photo illustration by Salon/Crown Publishing/Emory University)
The Whiteness Of Wealth by Dorothy A. Brown (Photo illustration by Salon/Crown Publishing/Emory University)

A typical white family has eight times the wealth of a typical Black family in the United States. Let that sink in for a long moment. The why behind that racial wealth gap has included a discussion of various factors, but until recently has overlooked one driving reason: The U.S. tax code. That's why Dorothy Brown, a professor of law at Emory University, wrote her compelling new book, "The Whiteness of Wealth: How the Tax System Impoverishes Black Americans — and How We Can Fix It."

The disparate impact our tax code has had on white and Black Americans has never been explored in this depth because — as Brown discussed in our recent "Salon Talks" conversation, the IRS doesn't collect data on the race of taxpayers. As a result, Brown became a "detective," as she put it, to uncover the real-world impact of our tax code on Americans by race.

What she discovered is jaw-dropping. Simply put, when white and Black Americans engage in the exact same thing — from marriage to homeownership to paying for college — U.S. tax policy typically provides far more advantages to whites than it does to Blacks.

For example, single wage-earning couples see a tax reduction when they get married, but couples who earn close to equal wages pay higher taxes upon tying the knot. On its face that doesn't seem like a big deal, but guess which couples are far more likely to be two-income, equal wage earners? Black families, as Brown details — including her own parents.

Our tax code also contributes to the fact that 73% of white people are homeowners, compared to only 44% of Black people.  As Brown documents, the tax code has fostered this gap by offering tax deductions when a person sells a home for a profit, but not one if you sell for a loss. Again, that provision is neutral on its face, but in the real world, far more white homeowners see their residences appreciate in value, given that they're likely to buy homes in white neighborhoods. In contrast, Black people purchase homes in Black or mixed neighborhoods and often don't see a significant increase in value, or any at all — which Brown has dubbed "the appreciation gap."

Brown, says she "veers between pessimism and optimism" as to whether the issues she has raised will be rectified. In any event, her work is a necessary first step on the road to addressing the institutional racism embedded in our tax code.

Watch the "Salon Talks" interview with Dorothy Brown here or read the transcript below, lightly edited for length and clarity.

Let's talk about your book. For me, the takeaway is that Black and white Americans could be doing the same thing in terms of employment, schooling and home ownership, but the impact of our tax code is not equal. 

That's exactly right. And when I decided to write about race and tax, I had no idea that the IRS did not collect or publish statistics by race. I thought this was going to be a lot easier. And 25 years later, I have the book, right? So what I had to do was become a detective of sorts. What got me started thinking about marriage for example, is I saw a Commission on Civil Rights report that had the following sentence, "Married Black wives contribute 40% of household income compared to married white wives that contribute 29%." Anybody else reading that statistic, it means nothing to them. To me? I struggled, because to me it explained why my parents paid so much in taxes. They paid so much in taxes because they were married to each other and earned roughly equal amounts.

You lay that out in your book. So why is that?

Right, because as I like to say, taxpayers bring their racial identities onto their tax returns. So notwithstanding a race-neutral law, we live in a world with systemic racism. So let's take jobs. We know the statistics that say Blacks don't earn what whites do. We know the statistics that show if you are a Black college graduate, it's harder for you to get an interview and when you do, you are targeted for lower-paying jobs. When we talk about married Black couples, it takes two owners to equal what one potential white male owner would get. So the job market and the racism in the job market impacts the level of taxes we pay, and impacts the exclusions we get. Think about employer-provided retirement accounts — those only come with the best jobs. Who is most likely to get those? And even when Black Americans get those jobs, research shows that Black college graduates are more likely to send money home to their parents and grandparents who didn't have the same opportunities, whereas white college graduates are more likely to get money from their parents and grandparents.

So think about these two workers, one Black, one white, they're next door to each other, or they're next cubicle to each other, pre-pandemic, right? And they are both eligible for a retirement account. The white employee maxes out because he doesn't have all of the burdens of his family. The Black college graduate on the other hand, sending money to their parents or grandparents, they have less available to save for retirement. If they do manage to get a retirement account and put money into it, you're more likely to take an early withdrawal, which has significant tax penalties associated with it. So Black Americans are less likely to have jobs that come with that tax-free retirement benefit. And even when they do, they're more likely to take money out, subjecting them to a tax penalty.

You make a point about your own parents. They're making relatively equal incomes, which from an outside point of view, people might go, "Oh, that's nice. Look, a man and a woman making relatively equal income." The problem is that's not what our tax code favors or incentivizes. Share a little bit about your own family, which I think is instructive in the bigger picture.

Yes. So my mother was a nurse, my father was a plumber and like any good daughter who has an LLM in tax, I started doing their tax returns. I would do mine and I would do theirs, and I made by myself what my parents made combined. And under our progressive tax system, I should be paying a whole lot more than they were. I wasn't, and I couldn't figure it out. It wasn't that I was doing anything wrong, I know what I'm doing. What I never could figure out until I became a professor and started studying this, was they paid higher taxes because their incomes were close together. Let's say my parents' combined income was $70,000. Take one $70,000 wage-earner who gets married, they get a tax cut. My parents, on the other hand — they get married, and their taxes go up. So the joint return that's so race-neutral is designed to benefit the way more white Americans do marriage than the way Black Americans do marriage.

Let's talk about another big thing in America, which is home ownership. And as you know, in your book, 73% of white people own homes but only 44% of Black people, for various reasons. How is the tax code implicated in this?

Yes. So everybody knows about the mortgage interest deduction, but let's start with tax subsidies for home ownership, which historically as well as today have been about helping white taxpayers because homeowners have always been disproportionately white. So the minute you have tax subsidies for home ownership, no matter what form it takes, you are helping white Americans. We know about the mortgage as a deduction, but that's less of a big deal these days. With the Trump tax cuts and more people taking the standard deduction, only like one in 10 Americans itemize anymore. And you can't get a mortgage as a deduction tax benefit if you don't itemize — that's not where the money is. The money with respect to tax subsidies for home ownership is how you can sell your home at a gain. If you're married, you get to keep half a million dollars of gain, tax free.

However, if you sell your home at a loss, no tax deduction for you. So your listeners are saying, "Well, what's the problem? When Black homeowners sell a home at a gain, they get tax-free gain. And that's no problem because nobody gets a loss." Now here's what you learn: The market for home ownership is based on white preferences. Where is the most home ownership appreciation? In virtually all-white neighborhoods, neighborhoods with very few Black Americans. Most white homeowners live in those neighborhoods. Most Black homeowners live in racially diverse or all Black neighborhoods. Therefore, when a Black homeowner sells, they are not going to get as much gain tax-free and worse, who's most likely to sell their home at a loss? Research shows that's Black homeowners. So not only do we get less gain tax free, we are more likely to get a non-deductible loss.

You go through some of the factors for why homes in Black communities might be lesser valued, You had, first of all, government-approved discrimination. Then you had redlining. But there's a third thing you bring up that the political left fails to acknowledge. Please share what they're missing.

What a lot of the left misses is that race discrimination isn't just historical, it's 21st-century race discrimination based on where white homeowners choose to live. And where they choose to live is in virtually all-white neighborhoods or neighborhoods with very few Black Americans. Now, when I say this, left law professors, whoever I'm talking to, they don't like to hear this and they often push back: "Well, Dorothy, this really isn't because white people are racist. What white people are really concerned about is property value." My response is, "As a Black homeowner, I don't care whether you're making a decision because you're revealing you're a racist, or you're just acting like one. My home value is based on your decision, on who you do not want to live next to."

Research shows videos of neighborhoods where the only thing that was changed were the actors hired to walk in the neighborhood. When it was all white, when it was all Black, and when it was 60% white and 40% Black. What did the viewers say? The white people who viewed it picked the all-white neighborhood. The Black viewers picked racially diverse or all Black neighborhoods. There was no fear of crime, it was an identical neighborhood. There was no social amenity gap, it was an identical neighborhood. So we see white Americans are comfortable in neighborhoods with no Black Americans, and that's a choice they make when they could choose to make different choices.

The last bit here you touch on is education, and how the tax code here also causes a racial disparity. 

How we pay for college is very different for Black Americans and white Americans. Black Americans are more likely to pay for college with debt. And the statistic that depressed me the most in this book is this statistic that shows 60% of Black Americans who start college don't graduate. So you have a significant amount of Black folks who don't graduate, but they leave with a lot of debt. So what does our tax law do with this debt? It allows an interest deduction up to $2,500, which when you look at the average Black debt versus white debt, the $2,500 of interest is enough for the average white debt, but not the average Black debt. So we actually have tax law, once again, that is advantaging how white Americans pay for college versus how Black Americans pay for college. And the other way white Americans have college paid for them, is with parents and grandparents and gifts. Black Americans simply don't have the wealth in their family to be able to do that.

So I'm sure people are wondering, "Well, are you saying the tax code was designed to help white people over Black people?" Is there an intent that you found?

So it's interesting, because I get this question a lot, and I get this question from white Americans. I never get this question from Black Americans. So it's really, really interesting. My first response is, well, look at the time when these provisions came into effect. Let's talk about the exclusion from home ownership. That dates back to 1951. Do we need any legislative history that says we didn't want to discriminate against Black people in 1951? No, because that was the norm. It was the law. That's what people did. In fact, 1950 was like the first time the majority of white Americans became homeowners, and suddenly we have a provision that they can sell without paying taxes.

So a lot of these provisions date back to Jim Crow, 1951, before Brown v. Board of Education. I haven't found a smoking gun: Blacks pay more. But if you look at the context, which is what history does, right? You look at the context within which these provisions were enacted, and it was obviously to benefit white people and not Black people. At the end of the day, whether it was intentional or not, Black Americans are paying higher taxes and we need to fix it.

What do you think about the idea of some form of reparations?

Oh, absolutely. I've shown how the tax code has stripped wealth from Black families by having them pay higher taxes. We need to be compensated for that. If we're talking about cash payments, before any cash payments are made, we need to fix the system for building wealth. Because if we don't do that, every Black person in America could get $100,000 tomorrow and in this system, before long it would wind up in white hands. So we have to fix the system for building wealth before we start talking about the specifics of reparations, but absolutely Black Americans need reparations.

Share a little bit about what suggestions you have that sort of begin to level the playing field as much as possible, without actually changing the laws?

So I urge Black Americans to be very intentional about what they do. Let's talk especially about home ownership. So we know the research that says, if you buy in an all-white neighborhood, you're going to make money and you're going to make a lot of money. If you buy in an all-Black or racially diverse neighborhood, you're not going to make as much money and you're more likely to get a loss if you sell. So what do you do? Do you buy in the all-white neighborhood, but deal with racist neighbors? So this is the problem. One of your neighbors could call the cops on you. If you have children, you're going to have to fight with the school administrators, who are going to tag your children as discipline problems when they're doing exactly what their white peers are doing.

So yeah, it's a good financial investment, but it's a freaking hassle. So say, "No, no, no. I'm going to live around Black people. So I know I'm not going to get as much out of my house." No. 1, do not put all your money to buy that house in the all-Black neighborhood or racially diverse neighborhood. No. 2, do not take a home equity loan. Take the money that you saved from not buying the biggest house you can afford and put it in your retirement account, put it in the stock market, put it in a child savings account. Do other things that will reap a benefit, rather than home ownership in an all-Black or racially diverse neighborhood. So it's just being intentional.

I read an interview where you said you were recently contacted by the Biden administration about this. What can you share?

Well, first of all, we all know that the first executive order that President Biden signed talked about racial equity across the government. We also created, in that executive order, a working data group, including the Department of the Treasury, which is where the tax statistics would come from. So I believe I got the call from the Biden administration because I've been fairly consistently pessimistic about the personnel at Treasury assigned to get this done. They have no background on talking about raising tax. So you have this directive from the top, but the people that the Biden administration has put in Treasury are not competent, in my view, to get this done.

So I got a call, I believe, so that lines of communication were open, so I could talk about it, and I did. I shared my extreme pessimism on getting this recent tax data done. And we committed to have ongoing dialogue. So I'm actually optimistic that we might see race and tax statistics from this administration. I'm also optimistic that we might start to see race conversations around tax like we've never seen before. But I'm going to keep pushing from the outside. Because as I said to the person I was talking to on the phone, I believe in "Trust, but verify." You're talking a good game, but you got to make good or I'm just going to keep talking about it.

In your book, you talk about varying between pessimism and optimism. Where do you stand right now?

I'm in the middle. I'm a lawyer and I know talk is cheap. I know lawyers are really good at using lots of words and saying nothing. Honestly, until I see some movement from the administration, I'm more toward the pessimism side than the optimism, but the dialogue continues. I will tell you I've been contacted by Congress. There's definitely reception, more reception on the part of Congress than I've seen from the Biden administration on race in tax. So I'm excited about that.

By Dean Obeidallah

Dean Obeidallah hosts the daily national SiriusXM radio program, "The Dean Obeidallah Show" on the network's progressive political channel. He is also a columnist for The Daily Beast and contributor to Opinion. He co-directed the comedy documentary "The Muslims Are Coming!" and is co-creator of the annual New York Arab American Comedy Festival. Follow him on Twitter @DeanObeidallah and Facebook @DeanofRadio

MORE FROM Dean Obeidallah

Related Topics ------------------------------------------

Authors Books Dorothy Brown Interviews Racism Salon Talks Salontv Taxes The Whiteness Of Wealth Wealth Gap