Big data could end professor lectures

Schools are embracing technology that tailors content to students' abilities, leaving teachers as roving mentors

Topics: Scientific American, Big Data, math, University, student loans, College, , ,

Big data could end professor lectures
This article was originally published by Scientific American.

Scientific American When Arnecia Hawkins enrolled at Arizona State University last fall, she did not realize she was volunteering as a test subject in an experimental reinvention of American higher education. Yet here she was, near the end of her spring semester, learning math from a machine. In a well-appointed computer lab in Tempe, on Arizona State’s desert resort of a campus, she and a sophomore named Jessica were practicing calculating annuities. Through a software dashboard, they could click and scroll among videos, text, quizzes and practice problems at their own pace. As they worked, their answers, along with reams of data on the ways in which they arrived at those answers, were beamed to distant servers. Predictive algorithms developed by a team of data scientists compared their stats with data gathered from tens of thousands of other students, looking for clues as to what Hawkins was learning, what she was struggling with, what she should learn next and how, exactly, she should learn it.

Having a computer for an instructor was a change for Hawkins. “I’m not gonna lie—at first I was really annoyed with it,” she says. The arrangement was a switch for her professor, too. David Heckman, a mathematician, was accustomed to lecturing to the class, but he had to take on the role of a roving mentor, responding to raised hands and coaching students when they got stumped. Soon, though, both began to see some benefits. Hawkins liked the self-pacing, which allowed her to work ahead on her own time, either from her laptop or from the computer lab. For Heckman, the program allowed him to more easily track his students’ performance. He could open a dashboard that told him, in granular detail, how each student was doing—not only who was on track and who was not but who was working on any given concept. Heckman says he likes lecturing better, but he seems to be adjusting. One definite perk for instuctors: the software does most of the grading for them.

At the end of the term, Hawkins will have completed the last college math class she will probably ever have to take. She will think back on this data-driven course model—so new and controversial right now—as the “normal” college experience. “Do we even have regular math classes here?” she asks.

Big Data Takes Education

Arizona State’s decision to move to computerized learning was born, at least in part, of necessity. With more than 70,000 students, Arizona State is the largest public university in the U.S. Like institutions at every level of American education, it is going through some wrenching changes. The university has lost 50 percent of its state funding over the past five years. Meanwhile enrollment is rising, with alarmingly high numbers of students showing up on campus unprepared to do college-level work. “There is a sea of people we’re trying to educate that we’ve never tried to educate before,” says Al Boggess, director of the Arizona State math department. “The politicians are saying, ‘Educate them. Remediation? Figure it out. And we want them to graduate in four years. And your funding is going down, too.’”

Two years ago Arizona State administrators went looking for a more efficient way to shepherd students through basic general-education requirements—particularly those courses, such as college math, that disproportionately cause students to drop out. A few months after hearing a pitch by Jose Ferreira, the founder and CEO of the New York City adaptive-learning start-up Knewton, Arizona State made a big move. That fall, with little debate or warning, it placed 4,700 students into computerized math courses. Last year some 50 instructors coached 7,600 Arizona State students through three entry-level math courses running on Knewton software. By the fall of 2014 ASU aims to adapt six more courses, adding another 19,000 students a year to the adaptive-learning ranks. (In May, Knewton announced a partnership with Macmillan Education, a sister company to Scientific American.)

Arizona State is one of the earliest, most aggressive adopters of data-driven, personalized learning. Yet educational institutions at all levels are pursuing similar options as a way to cope with rising enrollments, falling budgets and more stringent requirements for student achievement. Public primary and secondary schools in 45 states and the District of Columbia are rushing to implement new, higher standards in English-language arts and mathematics known as the Common Core state standards, and those schools need new instructional materials and tests to make that happen. Around half of those tests will be online and adaptive, meaning that a computer will tailor questions to each student’s ability and calculate each student’s score [see “Why We Need High-Speed Schools,” on page 69]. School systems are experimenting with a range of other adaptive programs, from math and reading lessons for elementary school students to “quizzing engines” that help high school students prepare for Advanced Placement exams. The technology is also catching on overseas. The 2015 edition of the Organization for Economic Co-operation and Development’s Program for International Student Assessment (PISA) test, which is given to 15-year-olds (in more than 70 nations and economies so far) every three years, will include adaptive components to evaluate hard-to-measure skills such as collaborative problem solving.

Proponents of adaptive learning say that technology has finally made it possible to deliver individualized instruction to every student at an affordable cost—to discard the factory model that has dominated Western education for the past two centuries. Critics say it is data-driven learning, not traditional learning, that threatens to turn schools into factories. They see this increasing digitization as yet another unnecessary sellout to for-profit companies that push their products on teachers and students in the name of “reform.” The supposedly advanced tasks that computers can now barely pull off—diagnosing a student’s strengths and weaknesses and adjusting materials and approaches to suit individual learners—are things human teachers have been doing well for hundreds of years. Instead of delegating these tasks to computers, opponents say, we should be spending more on training, hiring and retaining good teachers.

And while adaptive-learning companies claim to have nothing but the future of America’s children in mind, there is no denying the potential for profit. Dozens of them are rushing to get in on the burgeoning market for instructional technology, which is now a multibillion-dollar industry [see box at left]. As much as 20 percent of instructional content in K–12 schools is already delivered digitally, says Adam Newman, a founding partner of the market-analysis firm Education Growth Advisors. Although adaptive-learning software makes up only a small slice of the digital-instruction pie—around $50 million for the K–12 market—it could grow quickly. Newman says the concept of adaptivity is already very much in the water in K–12 schools. “In K–12, the focus has been on differentiating instruction for years,” he says. “Differentiating instruction, even without technology, is really a form of adaptation.”

Higher-education administrators are warming up to adaptivity, too. In a recent Inside Higher Ed/Gallup poll, 66 percent of college presidents said they found adaptive-learning and testing technologies promising. The Bill & Melinda Gates Foundation has launched the Adaptive Learning Market Acceleration Program, which will issue 10 $100,000 grants to U.S. colleges and universities to develop adaptive courses that enroll at least 500 students over three semesters. “In the long term—20 years out—I would expect virtually every course to have an adaptive component of some kind,” says Peter Stokes, an expert on digital education at Northeastern University. That will be a good thing, he says—an opportunity to apply empirical study and cognitive science to education in a way that has never been done. In higher education in particular, “very, very, very few instructors have a formal education in how to teach,” he says. “We do things, and we think they work. But when you start doing scientific measurement, you realize that some of our ways of doing things have no empirical basis.”

The Science of Adaptivity

In general, “adaptive” refers to a computerized-learning interface that constantly assesses a student’s thinking habits and automatically customizes material for him or her. Not surprisingly, though, competitors argue ferociously about who can claim the title of true adaptivity. Some say that a test that does nothing more than choose your next question based on whether you get the item in front of you correct—a test that steers itself according to the logic of binary branching—does not, in 2013, count as fully adaptive. In this view, adaptivity requires the creation of a psychometric profile of each user, plus the continuous adjustment of the experience based on that person’s progress.

To make this happen, adaptive-software makers must first map the connections among every concept in a piece of learning material. Once that is done, every time a student watches a video, reads an explanation, solves a practice problem or takes a quiz, data on the student’s performance, the effectiveness of the content, and more flow to a server. Then the algorithms take over, comparing that student with thousands or even millions of others. Patterns should emerge. It could turn out that a particular student is struggling with the same concept as students who share a specific psychometric profile. The software will know what works well for that type of student and will adjust the material accordingly. With billions of data points from millions of students and given enough processing power and experience, these algorithms should be able to do all kinds of prognostication, down to telling you that you will learn exponents best between 9:42 and 10:03 a.m.

They should also be able to predict the best way to get you to remember the material you are learning. Ulrik Juul Christensen, CEO of Area9, the developer of the data-analysis software underpinning McGraw-Hill’s adaptive LearnSmart products, emphasizes his company’s use of the concept of memory decay. More than two million students currently use LearnSmart’s adaptive software to study dozens of topics, either on their own or as part of a course. Research has shown that those students (all of us, really) remember a new word or fact best when they learn it and then relearn it when they are just on the cusp of forgetting it. Area9′s instructional software uses algorithms to predict each user’s unique memory-decay curve so that it can remind a student of something learned last week at the moment it is about to slip out of his or her brain forever.

Few human instructors can claim that sort of prescience. Nevertheless, Christensen dismisses the idea that computers could ever replace teachers. “I don’t think we are so stupid that we would let computers take over teaching our kids,” he says.


In March, Gerald J. Conti, a social studies teacher at Westhill High School in Syracuse, N.Y., posted a scathing retirement letter to his Facebook page that quickly became a viral sensation. “In their pursuit of Federal tax dollars,” he wrote, “our legislators have failed us by selling children out to private industries such as Pearson Education,” the educational-publishing giant, which has partnered with Knewton to develop products. “My profession is being demeaned by a pervasive atmosphere of distrust, dictating that teachers cannot be permitted to develop and administer their own quizzes and tests (now titled as generic ‘assessments’) or grade their own students’ examinations.” Conti sees big data leading not to personalized learning for all but to an educational monoculture: “STEM [science, technology, engineering and mathematics] rules the day, and ‘data driven’ education seeks only conformity, standardization, testing and a zombie-like adherence to the shallow and generic Common Core.”

Conti’s letter is only one example of the backlash building against tech-oriented, testing-focused education reform. In January teachers at Garfield High School in Seattle voted to boycott the Measures of Academic Progress (MAP) test, administered in school districts across the country to assess student performance. After tangling with their district’s superintendent and school board, the teachers continued the boycott, which soon spread to other Seattle schools. Educators in Chicago and elsewhere held protests to show solidarity. In mid-May it was announced that Seattle high schools would be allowed to opt out of MAP, as long as they replaced it with some other evaluation.

It would be easy for proponents of data-driven learning to counter these protests if they could definitely prove that their methods work better than the status quo. But they cannot do that, at least not yet. Empirical evidence about effectiveness is, as Darrell M. West, an adaptive-learning proponent and founder of the Brookings Institution’s Center for Technology Innovation, has written, “preliminary and impressionistic.” Any accurate evaluation of adaptive-learning technology would have to isolate and account for all variables: increases or decreases in a class’s size; whether the classroom was “flipped” (meaning homework was done in class and lectures were delivered via video on the students’ own time); whether the material was delivered via video, text or game; and so on. Arizona State says 78 percent of students taking the Knewton-ized developmental math course passed, up from 56 percent before. Yet it is always possible that more students are passing not because of technology but because of a change in policy: the university now lets students retake developmental math or stretch it over two semesters without paying tuition twice.

Even if proponents of adaptive technology prove that it works wonderfully, they will still have to contend with privacy concerns. It turns out that plenty of people find pervasive psychometric-data gathering unnerving. Witness the fury that greeted inBloom earlier this year. InBloom essentially offers off-site digital storage for student data—names, addresses, phone numbers, attendance, test scores, health records—formatted in a way that enables third-party education applications to use it. When inBloom was launched in February, the company announced partnerships with school districts in nine states, and parents were outraged. Fears of a “national database” of student information spread. Critics said that school districts, through inBloom, were giving their children’s confidential data away to companies who sought to profit by proposing a solution to a problem that does not exist. Since then, all but three of those nine states have backed out.

This might all seem like overreaction, but to be fair, adaptive-education proponents already talk about a student’s data-generated profile following them throughout their educational career and even beyond. Last fall the education-reform campaign Digital Learning Now released a paper arguing for the creation of “data backpacks” for pre-K–12 students—electronic transcripts that kids would carry with them from grade to grade so that they will show up on the first day of school with “data about their learning preferences, motivations, personal accomplishments, and an expanded record of their achievement over time.” Once it comes time to apply for college or look for a job, why not use the scores stored in their data backpacks as credentials? Something similar is already happening in Japan, where it is common for managers who have studied English with the adaptive-learning software iKnow to list their iKnow scores on their resumes.

This Is Not a Test

It is far from clear whether concerned parents and scorned instructors are enough to stop the march of big data on education. “The reality is that it’s going to be done,” says Eva Baker, director of the Center for the Study of Evaluation at the University of California, Los Angeles. “It’s not going to be a little part. It’s going to be a big part. And it’s going to be put in place partly because it’s going to be less expensive than doing professional development.”

That does not mean teachers are going away. Nor does it mean that schools will become increasingly test-obsessed. It could mean the opposite. Sufficiently advanced testing is indistinguishable from instruction. In a fully adaptive classroom, students will be continually assessed, with every keystroke and mouse click feeding a learner profile. High-stakes exams could eventually disappear, replaced by the calculus of perpetual monitoring.

Long before that happens, generational turnover could make these computerized methods of instruction and testing, so foreign now, unremarkable, as they are for Arizona State’s Hawkins and her classmates. Teachers could come around, too. Arizona State’s executive vice provost Phil Regier believes they will, at least: “I think a good majority of the instructors would say this was a good move. And by the way, in three years 80 percent of them aren’t going to know anything else.”

Seth Fletcher is an editorial assistant at Men's Journal.

More Related Stories

Featured Slide Shows

  • Share on Twitter
  • Share on Facebook
  • 1 of 11
  • Close
  • Fullscreen
  • Thumbnails

    Ten spectacular graphic novels from 2014

    Beautiful Darkness by Fabien Vehlmann & Kerascoët
    Kerascoët's lovely, delicate pen-and-watercolor art -- all intricate botanicals, big eyes and flowing hair -- gives this fairy story a deceptively pretty finish. You find out quickly, however, that these are the heartless and heedless fairies of folk legend, not the sentimental sprites beloved by the Victorians and Disney fans. A host of tiny hominid creatures must learn to survive in the forest after fleeing their former home -- a little girl who lies dead in the woods. The main character, Aurora, tries to organize the group into a community, but most of her cohort is too capricious, lazy and selfish to participate for long. There's no real moral to this story, which is refreshing in itself, beyond the perpetual lessons that life is hard and you have to be careful whom you trust. Never has ugly truth been given a prettier face.

    Ten spectacular graphic novels from 2014

    Climate Changed: A Personal Journey Through the Science by Philippe Squarzoni
    Squarzoni is a French cartoonist who makes nonfiction graphic novels about contemporary issues and politics. While finishing up a book about France under Jacques Chirac, he realized that when it came to environmental policy, he didn't know what he was talking about. "Climate Changed" is the result of his efforts to understand what has been happening to the planet, a striking combination of memoir and data that ruminates on a notoriously elusive, difficult and even imponderable subject. Panels of talking heads dispensing information (or Squarzoni discussing the issues with his partner) are juxtaposed with detailed and meticulous yet lyrical scenes from the author's childhood, the countryside where he takes a holiday and a visit to New York. He uses his own unreachable past as a way to grasp the imminent transformation of the Earth. The result is both enlightening and unexpectedly moving.

    Ten spectacular graphic novels from 2014

    Here by Richard McGuire
    A six-page version of this innovative work by a regular contributor to the New Yorker first appeared in RAW magazine 25 years ago. Each two-page spread depicts a single place, sometimes occupied by a corner of a room, over the course of 4 billion years. The oldest image is a blur of pink and purple gases; others depict hazmat-suited explorers from 300 years in the future. Inset images show the changing decor and inhabitants of the house throughout its existence: family photos, quarrels, kids in Halloween costumes, a woman reading a book, a cat walking across the floor. The cumulative effect is serene and ravishing, an intimation of the immensity of time and the wonder embodied in the humblest things.

    Ten spectacular graphic novels from 2014

    Kill My Mother by Jules Feiffer
    The legendary Pulitzer Prize-winning cartoonist delivers his debut graphic novel at 85, a deliriously over-the-top blend of classic movie noir and melodrama that roams from chiaroscuro Bay City to Hollywood to a USO gig in the Pacific theater of World War II. There's a burnt-out drunk of a private eye, but the story is soon commandeered by a multigenerational collection of ferocious women, including a mysterious chanteuse who never speaks, a radio comedy writer who makes a childhood friend the butt of a hit series and a ruthless dame intent on making her whiny coward of a husband into a star. There are disguises, musical numbers and plenty of gunfights, but the drawing is the main attraction. Nobody convey's bodies in motion more thrillingly than Feiffer, whether they're dancing, running or duking it out. The kid has promise.

    Ten spectacular graphic novels from 2014

    The Motherless Oven by Rob Davis
    This is a weird one, but in the nervy surreal way that word-playful novels like "A Clockwork Orange" or "Ulysses" are weird. The main character, a teenage schoolboy named Scarper Lee, lives in a world where it rains knives and people make their own parents, contraptions that can be anything from a tiny figurine stashable in a pocket to biomorphic boiler-like entities that seem to have escaped from Dr. Seuss' nightmares. Their homes are crammed with gadgets they call gods and instead of TV they watch a hulu-hoop-size wheel of repeating images that changes with the day of the week. They also know their own "death day," and Scarper's is coming up fast. Maybe that's why he runs off with the new girl at school, a real troublemaker, and the obscurely dysfunctional Castro, whose mother is a cageful of talking parakeets. A solid towline of teenage angst holds this manically inventive vision together, and proves that some graphic novels can rival the text-only kind at their own game.

    Ten spectacular graphic novels from 2014

    NOBROW 9: It's Oh So Quiet
    For each issue, the anthology magazine put out by this adventurous U.K.-based publisher of independent graphic design, illustration and comics gives 45 artists a four-color palette and a theme. In the ninth issue, the theme is silence, and the results are magnificent and full of surprises. The comics, each told in images only, range from atmospheric to trippy to jokey to melancholy to epic to creepy. But the two-page illustrations are even more powerful, even if it's not always easy to see how they pertain to the overall concept of silence. Well, except perhaps for the fact that so many of them left me utterly dumbstruck with visual delight.

    Ten spectacular graphic novels from 2014

    Over Easy by Mimi Pond
    When Pond was a broke art student in the 1970s, she took a job at a neighborhood breakfast spot in Oakland, a place with good food, splendid coffee and an endlessly entertaining crew of short-order cooks, waitresses, dishwashers and regular customers. This graphic memoir, influenced by the work of Pond's friend, Alison Bechdel, captures the funky ethos of the time, when hippies, punks and disco aficionados mingled in a Bay Area at the height of its eccentricity. The staff of the Imperial Cafe were forever swapping wisecracks and hopping in and out of each other's beds, which makes them more or less like every restaurant team in history. There's an intoxicating esprit de corps to a well-run everyday joint like the Imperial Cafe, and never has the delight in being part of it been more winningly portrayed.

    Ten spectacular graphic novels from 2014

    The Shadow Hero by Gene Luen Yang and Sonny Liew
    You don't have to be a superhero fan to be utterly charmed by Yang and Liew's revival of a little-known character created in the 1940s by the cartoonist Chu Hing. This version of the Green Turtle, however, is rich in characterization, comedy and luscious period detail from the Chinatown of "San Incendio" (a ringer for San Francisco). Hank, son of a mild-mannered grocer, would like to follow in his father's footsteps, but his restless mother (the book's best character and drawn with masterful nuance by Liew) has other ideas after her thrilling encounter with a superhero. Yang's story effortlessly folds pathos into humor without stooping to either slapstick or cheap "darkness." This is that rare tribute that far surpasses the thing it celebrates.

    Ten spectacular graphic novels from 2014

    Shoplifter by Michael Cho
    Corinna Park, former English major, works, unhappily, in a Toronto advertising agency. When the dissatisfaction of the past five years begins to oppress her, she lets off steam by pilfering magazines from a local convenience store. Cho's moody character study is as much about city life as it is about Corinna. He depicts her falling asleep in front of the TV in her condo, brooding on the subway, roaming the crowded streets after a budding romance goes awry. Like a great short story, this is a simple tale of a young woman figuring out how to get her life back, but if feels as if it contains so much of contemporary existence -- its comforts, its loneliness, its self-deceptions -- suspended in wintery amber.

    Ten spectacular graphic novels from 2014

    Through the Woods by Emily Carroll
    This collection of archetypal horror, fairy and ghost stories, all about young girls, comes lushly decked in Carroll's inky black, snowy white and blood-scarlet art. A young bride hears her predecessor's bones singing from under the floorboards, two friends make the mistake of pretending to summon the spirits of the dead, a family of orphaned siblings disappears one by one into the winter nights. Carroll's color-saturated images can be jagged, ornate and gruesome, but she also knows how to chill with absence, shadows and a single staring eye. Literary readers who cherish the work of Kelly Link or the late Angela Carter's collection, "The Bloody Chamber," will adore the violent beauty on these pages.

  • Recent Slide Shows


Loading Comments...