Testing, testing

Critics of the new writing section of the SAT say that the only thing the test measures is a student's aptitude for B.S.

Topics:

Testing, testing

What fresh hell is this? The newly expanded version of the Scholastic Aptitude Test (SAT), first offered in March, has three sections: math, verbal and really verbal. On top of filling in row after row of infernal unblinking ovals — many of them in a whole new section on English grammar and usage — students must also write an impromptu essay, from scratch, in 25 minutes.

As it turns out, some students who took the test on May 7 did not feel entirely unprepared. “On the way to the test we had just heard about the MIT study that connected the length of the essay to the scores,” says Emily Rackleff, a junior at Newport High School in Newport, Ore. “So we all tried to write as much as possible.”

This “MIT study” is actually a series of calculations made by Les Perelman, a director of undergraduate writing at MIT who has developed and administered holistically graded writing tests for 25 years. (Full disclosure: He’s also been a family friend for at least that long.) Perelman, along with the National Council of Teachers of English, has concluded that the new essay section, contrary to the College Board’s intentions, is not only a lousy test of — but also a threat to — students’ writing skills.

While attending a recent panel on the new SAT, Perelman, acting on a hunch, reviewed more than 50 sample essays that the College Board had made public for students and graders. He ran some numbers (“because I am from MIT and that is what we do,” he says) and found a striking correlation between the length of the essays and their score. (For scoring, each of two graders gives a separate rank of 1 to 6, for a maximum total of 12.) “If you just looked at length and nothing else, you could be right in predicting the score over 90 percent of the time,” he says. (To prove his point, he even guessed the scores of sample essays just by gauging their length without reading them. He was 10 for 10.)

Perelman was also dismayed to learn that the College Board, which administers the SAT, has explicitly said — on the grounds that it is testing writing, not historical, literary, or scientific knowledge — that factual references in the essays needn’t be correct. If you misstate the date of the War of 1812, for example, the readers won’t mark you down.



“To write something that’s long and has the appearance of having a sound, detailed argument but really doesn’t make much sense: There is a term for that,” Perelman says delicately. And that is what is being rewarded.”

Perelman cites one of the sample essays the College Board provides for grader-training purposes: a composition that uses “Madame Bovary” as an example of the dangers of secrecy. It’s not awful, but it does include sentences such as “If secrecy were eradicated, many problems, such as internal division, but also possibly hate, might also be eliminated.” On the 1 to 6 scale, it serves as an example to scorers of a perfect 6.

Many students, of course, know exactly what they’re doing. “The essay topic was completely inane, the sort of thing I hate to write about,” says Sho Uemura, 16, a junior at Stuyvesant High School in New York City, of the SAT he took on May 7. “I don’t have any personal opinion on whether our lives are shaped by our jobs. I ended up writing a boring, formulaic five-paragraph essay that managed to fill most of the space. I got it done in about 20 minutes and drowsed for most of the remaining time.”

Of course, it’s no secret that bullshitting can go a long way toward filling the yawning blank space of a testing booklet (or that it’s a skill that serves students later in life). Nor was the SAT ever expected to delight students, or to reward creativity and insight. One might wonder, therefore, why people are getting so worked up over 25 minutes — or about 500 words — of a high school junior’s life. After all, the essay winds up counting for only a small portion of the total score — and many colleges are saying that at least for now, it’s not about to make or break a student’s chances. In fact, only 429 of about 1,600 four-year colleges have officially informed the College Board that they’ll require writing scores. Still, since 80 percent of colleges require the SAT for admissions, the real question is how they’ll use the new scores. “It’ll be beneficial to consider the essay as part of a holistic review of a student,” says Mark Hatch, vice president for Enrollment Management at Colorado College. “But it’s not something that colleges will be eager to integrate into their academic ranking system right away. We need to study it first.”

The main problem with the essay, say people like Perelman, is that it’s actually not just about those 25 minutes. Even the College Board says so. Chiara Coletti, vice president of public affairs, says that the organization has long been concerned that, especially in lower-income schools, writing instruction needs improvement. “By putting writing on the SAT, we believed we’d focus more attention on writing in schools from K through 12 and across the curriculum,” she says. But it’s the wrong kind of attention, say critics. “You’re getting teachers to train students to be bad writers,” Perelman says. The specter of the essay, he claims, could have a chilling effect on the learning and teaching of writing in high school, encouraging students only to perfect the art of the formulaic (to say nothing of wordy) five-paragraph “write for the teacher” essay.

“The SAT carries such weight that teachers will feel pressured to teach to the test: to help students prepare for the very specific kind of writing that the SAT asks them to do,” says Bob Yagelski, associate professor of English at SUNY Albany and chair of the National Council of Teachers of English Task Force on the SAT and ACT writing tests. In fact, in a recent report, the NCTE charged that the SAT’s essay test threatens to “compromise student writers and undermine longstanding efforts to improve writing instruction in the nation’s schools.”

At very least, English teachers say, the essay puts them in a tricky position. “The purpose of academic writing is to get kids to think deeply and explore ideas in as broad a way as possible, to find an authentic voice,” says the head of the English department of a suburban Boston high school, who did not want to be identified. “But with 25 minutes to write, they’re not thinking broadly. They’re thinking, ‘Follow the formula.’ We could tell students, ‘OK, this is just what you do when you take the SAT. Normally we ask you to be broad and deep and thoughtful, but for this particular test you need to go in with this narrow approach’ — well, that makes us hypocrites.”

SAT coaches don’t necessarily have the same conscience issues as some teachers. “What we recommend is that you memorize the basic essay formula and a couple of facts about a topic you’re interested in, so that you pretty much know what you’re going to write before you take the test,” says Yale sophomore-to-be Janet Xu, guest editor of the latest edition of the SAT guide “Up Your Score: The Underground Guide to The SAT.” “This works because the SAT questions are usually very broad.”

Indeed. An SAT coach based in Rochester, N.Y., who works for one of the major test-prep outfits and didn’t want to be named, recently took the new test himself to see what he was up against. “I was like, OK, I know all about ‘The Scarlet Letter,’ I know all about Florence Nightingale, I know all about the tsunami,” he says. “The essay question was something like, Does work give life meaning? So, Florence Nightingale, her work was to heal people — that gives life meaning. The tsunami, the rescue efforts showed the meaning of volunteer work. ‘The Scarlet Letter’ — I’m sure there was some work in there somewhere … OK, the work of retribution gave Hester Prynne’s life meaning. If you have examples of anything, you can write any essay,” he says. “And you can always use Florence Nightingale, no matter what.”

In response to Perelman’s concerns, the College Board has issued a statement (“I’d give it about a 3,” says Perelman) insisting that there is — and should be — a general correlation between length and score. “We do find plenty of essays that are long and haven’t developed a point of view at all, and short ones that manage to accomplish an amazing amount, but it’s rare. I think that’s just the way it is,” Coletti says, noting that the trained graders, themselves teachers, can surely be relied upon to make the proper distinctions.

Coletti also asserts that writing on demand does have its place — that it’s a necessary skill for tackling not only the on-deadline workplace memo but also the college exam blue book. According to the colleges the board has surveyed, “When called upon to do this, students often display an alarming lack of skill,” she says. “A lot of them simply didn’t learn it.”

True enough, says Xu. Although she’s not enamored of the essay section itself, she sees the new SAT as “simply a recognition of the new direction that standardized testing is taking,” she says. “Instead of testing kids on whether ‘apple’ is to ‘fruit’ as ‘magnanimous’ is to ‘sycophancy,’ it’s actually testing them on skills they need in college! Gasp!”

Ask a student what he or she thought of the essay, though, and you’re unlikely to hear, “Well, it gave me the opportunity to develop and show my preparedness for college, not to mention the odd workplace memo.” According to a recent survey by Kaplan Test Prep, only 13 percent of test takers thought the essay was the hardest section of the test — but 55 percent of them said their scores would not represent their skills. Over 25 percent said they ran out of time and had to turn in unfinished work.

“It was really overwhelming,” says Andrew Elliman, 17, a junior at the Kent Denver School in Denver, Colo., who took the test in March. “The question was something like ‘Describe the importance of creativity in today’s world.’ If someone asked you that, you could think about it for years and not come up with a good answer. But 25 minutes? I’d never written an essay on such a broad topic in such a short time. It came out of nowhere, totally broadsided me, like a car accident.”

According to the SAT’s most cynical detractors, the College Board is torturing students like Andrew only as a means of self-preservation. The rumor is that the writing component was added only in a desperate effort to keep the University of California, a giant client, from dropping the test as an admissions requirement. “The motivation for the ‘new’ SAT ‘writing’ test — we always put ‘new’ and ‘writing’ in quotes — is that it’s a marketing ploy,” says Bob Schaeffer, public education director of the National Center for Fair and Open Testing, suggesting that the College Board basically grafted what used to be the SAT subject test in writing onto the “new” test. “The College Board took what had been an unpopular, optional test and used it to burnish their image and enhance their bottom line,” he says. Coletti adamantly denies these claims, stating that some sort of writing section has been in the works for at least a decade but has hit mainly technological speed bumps.

Whatever motives you may ascribe to those who inflict the SAT, one point is salient, says Perelman. “We’re one of the few developed countries in the world where such testing is done by private agencies with no supervision by a public or quasi-public organization. It’s very much as if we allowed drug companies to market drugs without the FDA. To quote Juvenal in his sixth Satire: Who watches the watchmen?”

“I could have told you that that was from Caesar’s ‘Gallic Wars,’” he adds. “But it wouldn’t have made any difference.”

Award-winning journalist Lynn Harris is author of the comic novel "Death by Chick Lit" and co-creator of BreakupGirl.net. She also writes for the New York Times, Glamour, and many others.

More Related Stories

Featured Slide Shows

  • Share on Twitter
  • Share on Facebook
  • 1 of 11
  • Close
  • Fullscreen
  • Thumbnails

    Ten spectacular graphic novels from 2014

    Beautiful Darkness by Fabien Vehlmann & Kerascoët
    Kerascoët's lovely, delicate pen-and-watercolor art -- all intricate botanicals, big eyes and flowing hair -- gives this fairy story a deceptively pretty finish. You find out quickly, however, that these are the heartless and heedless fairies of folk legend, not the sentimental sprites beloved by the Victorians and Disney fans. A host of tiny hominid creatures must learn to survive in the forest after fleeing their former home -- a little girl who lies dead in the woods. The main character, Aurora, tries to organize the group into a community, but most of her cohort is too capricious, lazy and selfish to participate for long. There's no real moral to this story, which is refreshing in itself, beyond the perpetual lessons that life is hard and you have to be careful whom you trust. Never has ugly truth been given a prettier face.

    Ten spectacular graphic novels from 2014

    Climate Changed: A Personal Journey Through the Science by Philippe Squarzoni
    Squarzoni is a French cartoonist who makes nonfiction graphic novels about contemporary issues and politics. While finishing up a book about France under Jacques Chirac, he realized that when it came to environmental policy, he didn't know what he was talking about. "Climate Changed" is the result of his efforts to understand what has been happening to the planet, a striking combination of memoir and data that ruminates on a notoriously elusive, difficult and even imponderable subject. Panels of talking heads dispensing information (or Squarzoni discussing the issues with his partner) are juxtaposed with detailed and meticulous yet lyrical scenes from the author's childhood, the countryside where he takes a holiday and a visit to New York. He uses his own unreachable past as a way to grasp the imminent transformation of the Earth. The result is both enlightening and unexpectedly moving.

    Ten spectacular graphic novels from 2014

    Here by Richard McGuire
    A six-page version of this innovative work by a regular contributor to the New Yorker first appeared in RAW magazine 25 years ago. Each two-page spread depicts a single place, sometimes occupied by a corner of a room, over the course of 4 billion years. The oldest image is a blur of pink and purple gases; others depict hazmat-suited explorers from 300 years in the future. Inset images show the changing decor and inhabitants of the house throughout its existence: family photos, quarrels, kids in Halloween costumes, a woman reading a book, a cat walking across the floor. The cumulative effect is serene and ravishing, an intimation of the immensity of time and the wonder embodied in the humblest things.

    Ten spectacular graphic novels from 2014

    Kill My Mother by Jules Feiffer
    The legendary Pulitzer Prize-winning cartoonist delivers his debut graphic novel at 85, a deliriously over-the-top blend of classic movie noir and melodrama that roams from chiaroscuro Bay City to Hollywood to a USO gig in the Pacific theater of World War II. There's a burnt-out drunk of a private eye, but the story is soon commandeered by a multigenerational collection of ferocious women, including a mysterious chanteuse who never speaks, a radio comedy writer who makes a childhood friend the butt of a hit series and a ruthless dame intent on making her whiny coward of a husband into a star. There are disguises, musical numbers and plenty of gunfights, but the drawing is the main attraction. Nobody convey's bodies in motion more thrillingly than Feiffer, whether they're dancing, running or duking it out. The kid has promise.

    Ten spectacular graphic novels from 2014

    The Motherless Oven by Rob Davis
    This is a weird one, but in the nervy surreal way that word-playful novels like "A Clockwork Orange" or "Ulysses" are weird. The main character, a teenage schoolboy named Scarper Lee, lives in a world where it rains knives and people make their own parents, contraptions that can be anything from a tiny figurine stashable in a pocket to biomorphic boiler-like entities that seem to have escaped from Dr. Seuss' nightmares. Their homes are crammed with gadgets they call gods and instead of TV they watch a hulu-hoop-size wheel of repeating images that changes with the day of the week. They also know their own "death day," and Scarper's is coming up fast. Maybe that's why he runs off with the new girl at school, a real troublemaker, and the obscurely dysfunctional Castro, whose mother is a cageful of talking parakeets. A solid towline of teenage angst holds this manically inventive vision together, and proves that some graphic novels can rival the text-only kind at their own game.

    Ten spectacular graphic novels from 2014

    NOBROW 9: It's Oh So Quiet
    For each issue, the anthology magazine put out by this adventurous U.K.-based publisher of independent graphic design, illustration and comics gives 45 artists a four-color palette and a theme. In the ninth issue, the theme is silence, and the results are magnificent and full of surprises. The comics, each told in images only, range from atmospheric to trippy to jokey to melancholy to epic to creepy. But the two-page illustrations are even more powerful, even if it's not always easy to see how they pertain to the overall concept of silence. Well, except perhaps for the fact that so many of them left me utterly dumbstruck with visual delight.

    Ten spectacular graphic novels from 2014

    Over Easy by Mimi Pond
    When Pond was a broke art student in the 1970s, she took a job at a neighborhood breakfast spot in Oakland, a place with good food, splendid coffee and an endlessly entertaining crew of short-order cooks, waitresses, dishwashers and regular customers. This graphic memoir, influenced by the work of Pond's friend, Alison Bechdel, captures the funky ethos of the time, when hippies, punks and disco aficionados mingled in a Bay Area at the height of its eccentricity. The staff of the Imperial Cafe were forever swapping wisecracks and hopping in and out of each other's beds, which makes them more or less like every restaurant team in history. There's an intoxicating esprit de corps to a well-run everyday joint like the Imperial Cafe, and never has the delight in being part of it been more winningly portrayed.

    Ten spectacular graphic novels from 2014

    The Shadow Hero by Gene Luen Yang and Sonny Liew
    You don't have to be a superhero fan to be utterly charmed by Yang and Liew's revival of a little-known character created in the 1940s by the cartoonist Chu Hing. This version of the Green Turtle, however, is rich in characterization, comedy and luscious period detail from the Chinatown of "San Incendio" (a ringer for San Francisco). Hank, son of a mild-mannered grocer, would like to follow in his father's footsteps, but his restless mother (the book's best character and drawn with masterful nuance by Liew) has other ideas after her thrilling encounter with a superhero. Yang's story effortlessly folds pathos into humor without stooping to either slapstick or cheap "darkness." This is that rare tribute that far surpasses the thing it celebrates.

    Ten spectacular graphic novels from 2014

    Shoplifter by Michael Cho
    Corinna Park, former English major, works, unhappily, in a Toronto advertising agency. When the dissatisfaction of the past five years begins to oppress her, she lets off steam by pilfering magazines from a local convenience store. Cho's moody character study is as much about city life as it is about Corinna. He depicts her falling asleep in front of the TV in her condo, brooding on the subway, roaming the crowded streets after a budding romance goes awry. Like a great short story, this is a simple tale of a young woman figuring out how to get her life back, but if feels as if it contains so much of contemporary existence -- its comforts, its loneliness, its self-deceptions -- suspended in wintery amber.

    Ten spectacular graphic novels from 2014

    Through the Woods by Emily Carroll
    This collection of archetypal horror, fairy and ghost stories, all about young girls, comes lushly decked in Carroll's inky black, snowy white and blood-scarlet art. A young bride hears her predecessor's bones singing from under the floorboards, two friends make the mistake of pretending to summon the spirits of the dead, a family of orphaned siblings disappears one by one into the winter nights. Carroll's color-saturated images can be jagged, ornate and gruesome, but she also knows how to chill with absence, shadows and a single staring eye. Literary readers who cherish the work of Kelly Link or the late Angela Carter's collection, "The Bloody Chamber," will adore the violent beauty on these pages.

  • Recent Slide Shows

Comments

0 Comments

Comment Preview

Your name will appear as username ( settings | log out )

You may use these HTML tags and attributes: <a href=""> <b> <em> <strong> <i> <blockquote>