Farhad Manjoo

How Comcast blocks your Internet traffic

The broadband company's audacious scheme to interrupt your data proves why network neutrality rules are crucial.

  • more
    • All Share Services

How Comcast blocks your Internet traffic

Bless the Associated Press for unearthing, through careful and diligent investigation, Comcast’s shameful, hidden Internet traffic-management scheme.

Comcast, the AP determined, actively manages data on its network by using software to essentially masquerade as its subscribers’ machines. When non-Comcast Internet subscribers request files from your Comcast-connected machine — as happens in peer-to-peer file-sharing applications — Comcast’s technology steps in and tells the non-Comcast subscriber you’re not available.

This is a difficult story to explain, but it’s quite important. For years, consumer advocates have been demanding that Congress and/or the Federal Communications Commission impose “network neutrality” regulations that would force broadband providers (like Comcast) to treat all data on a network equally. Lawmakers have so far failed to do so.

Broadband providers, meanwhile, insist that they do treat all traffic equally, but they reserve the right to use certain technologies to “manage” data on their network. The Comcast plan suggests that broadband providers mean something very broad by “traffic management” — including, it appears, purposefully stepping into your network sessions to shut them down.

To understand why this whole process is so egregious, let’s look at it in FAQ format.

What is Comcast doing, and why?

The Internet is awash in peer-to-peer applications. In such programs, you get (and send) pieces of a file from (and to) computers all over, rather than communicating with a single big server (as you do, say, when you download a YouTube video).

The technology first gained prominence with music- and movie-sharing apps — Napster, Kazaa, etc. — but today P2P tech is deployed in all kinds of software, including for Internet phone services (Skype) and TV (Joost). BitTorrent, one of the most popular peer-to-peer protocols, is used to download all kinds of stuff, both legal and illegal.

Broadband providers have a love-hate relationship with peer-to-peer apps. On the one hand, peer-to-peer programs increase the demand for high-speed access — many people decide to subscribe to broadband service only because of amazing apps like Kazaa, BitTorrent, Skype and others.

But peer-to-peer programs also eat up space on a network, because every user is downloading and uploading data for long periods of time. And for providers, a clogged network costs money and hurts their reputation. (If peer-to-peer users use up all the space, other users complain that their Web surfing is too slow.)

Providers thus have an incentive to reduce peer-to-peer traffic on their networks. But they can’t do so openly because, remember, a lot of people only pay for services like Comcast in order to use peer-to-peer programs.

Moreover, in their marketing copy, Comcast and other broadband companies play up the “unlimited” nature of their plans. They don’t really want to tell people that, actually, they’re managing their networks so that you can’t do all you want with it (though in the fine print that subscribers never read, they all reserve the right to do so).

The upshot, then: Comcast wants to manage its traffic. It just doesn’t want people to know that it does.

So how does Comcast silently manage traffic?

The effort that the AP reports on was first discovered by Robb Topolski, a software engineer who hangs out at the forums on DSLReports.com. In May, he posted a detailed note on the forum describing Comcast’s traffic management operation.

The system works, Topolski guessed, by limiting communication at the “boundary” of Comcast’s network — that is, the point where Comcast’s network connects with the larger Internet.

To detect peer-to-peer communication, Comcast inspects packets — the smallest meaningful bit of information on the Internet — as they cross the network boundary. If Comcast determines that there are too many peer-to-peer users within its network sending files to people outside the network, it begins to interrupt the connections between Comcast users and those beyond Comcast.

To interrupt these communications, Comcast appears to be using technology made by a network management company called Sandvine. What’s remarkable is how Sandvine manages to disrupt peer-to-peer traffic.

As Topolski describes it, Sandvine’s system sends a “forged” packet to each of the two computers engaged in a peer-to-peer transfer — the forged packet looks like it came from the other person’s computer, and it basically tells each machine that the other is unavailable, ending the transfer.

The AP describes this marvelously: “Each PC gets a message invisible to the user that looks like it comes from the other computer, telling it to stop communicating. But neither message originated from the other computer — it comes from Comcast. If it were a telephone conversation, it would be like the operator breaking into the conversation, telling each talker in the voice of the other: ‘Sorry, I have to hang up. Goodbye.’”

So what? Isn’t Comcast only stopping illegal file sharing?

No! Comcast’s system doesn’t look at the copyright status of the materials you’re trading — it only looks at the technical protocols you’re using to conduct the trade, and blocks access based on those protocols alone. And just because people use these technical protocols to trade illegal materials doesn’t mean that every use is illegal.

Case in point: To test how Comcast is managing traffic, AP reporters tried to download a version of the King James Bible using BitTorrent. The Bible, of course, is perfectly legal to trade; indeed, some people might say that putting the good book up for others to download is a blessed thing.

But when AP reporters tried to download the Bible from Comcast subscribers in Philadelphia and San Francisco, they found that the connections were either blocked outright or delayed. (Downloads from other providers worked fine.)

In his post, Rob Topolski points to another way Comcast’s system can disrupt legitimate traffic.

Say you have a band and you want to put up your CD on a file-sharing network for others to download. If you’re a Comcast subscriber, you would find this very hard to do — since Comcast limits peer-to-peer connections at the network boundary, “the time it would take to get a complete copy of a music file to a point outside of the Comcast network is dramatically increased,” Topolski wrote.

OK, but so what? Even if Comcast is blocking peer-to-peer traffic, that doesn’t affect me — all I’m doing is browsing the Web!

Sure, this only affects peer-to-peer transfers — at least, as far as we know. The most alarming thing about this scheme is that Comcast is conducting it on the sly. It didn’t alert anyone to its filtering mechanism — not its customers, not other ISPs, nobody.

Indeed, Comcast is still not coming clean. A company rep tells the AP: “We rarely disclose our vendors or our processes for operating our network for competitive reasons and to protect against network abuse,” he said.

And then there’s the sheer dishonesty of the practice. Comcast’s system is silently listening in to your Internet traffic and inserting itself into the communication in order to shut it down.

If the company feels justified doing this on peer-to-peer connections, what’s to say it wouldn’t feel similarly justified shutting down or slowing down your communication with Amazon.com, or NYTimes.com, or YouTube or any other online service (whether because it doesn’t like the content, or because it’s got an economic incentive, or because it’s just mean) — and all without telling us?

OK, so what can we do about this?

It’d be wonderful if the solution was to simply stop subscribing to Comcast. If that would make you feel better, by all means, cancel your subscription.

But know this: Other broadband vendors have not distinguished themselves on the issue of network neutrality. In general, major broadband companies say they should be free to manage traffic on their networks, and it’s impossible to tell how expansively they understand that “management” role.

If Comcast is saving money by adopting such methods, you can bet others are already doing so, or soon will. It would be shocking if Comcast were the only one.

But there is an obvious solution. It has been obvious for some time. We need a law!

Providers should be proscribed from interrupting customers’ connections or, at the very least, from doing so secretly — if they’re going to disrupt your traffic in any way, they should be forced to tell you how.

Broadband companies have long argued that network neutrality regulations are unnecessary. The Comcast scheme pretty definitively proves otherwise.

[Flickr picture by dmuth.]

The thinking man’s action hero

Using paper clips, chewing gum, chocolate and down-home ingenuity, MacGyver always saved the day. Let's bring him back -- and give him a girl!

  • more
    • All Share Services

The thinking man's action hero

It isn’t necessary to explain how, in the pilot episode of “MacGyver,” our mulleted, Midwestern hero gets himself trapped inside a top-secret research bunker overflowing with sulfuric acid. Suffice it to say, he needs to find a way out, and probably soon (because government agents are fixing to fire a missile at the bunker to prevent the acid from spilling into a nearby aquifer). Plus, he has to save the people he has found inside (among them a gun-wielding climate scientist who wants destroy the bunker in an effort to set back research into an ozone-layer-ruining weapon of mass destruction). Fortunately, MacGyver has a few chocolate bars, a scrap of sodium metal, a cold capsule, a pair of binoculars and cigarettes.

He uses the chocolate to plug up the leaking tank of acid — sulfuric acid reacts with sugar to form a kind of glue. The sodium, scraped into the shell of the cold capsule and splashed into a sealed bottle of water, makes for a handy time-delay bomb, which proves useful for blowing through a wall that blocks the group’s escape. The smoke from the cigarettes illuminates the bunker’s laser-beam security system that he has to get through to move through the bunker (no secret underground research lab is complete without lasers); MacGyver uses the binocular lens to aim the laser at its own control unit, shutting down the security system.

But how does he get out of the bunker? Oh, that’s the easy part: MacGyver finds a switch that controls the lights in an above-ground control tower. He flashes the lights on and off to send an SOS message in Morse code. The guys in the tower, realizing Mac’s in the bunker, alive, call off the missile — and for the first of 139 times during the show’s seven-year run from 1985 to 1992, MacGyver saves the day.

This first episode is nearly perfect. It neatly telegraphs MacGyver’s soul: A laid-back fellow oozing can-do heartland ingenuity, MacGyver is handsome but dorky, charming but self-effacing, a friend to orphans and children with disabilities, tolerant of people from foreign lands, and though he has every opportunity for indiscretion, he’s always a gentleman around women. MacGyver, played by the affable Richard Dean Anderson, works as a secret agent for a vaguely defined defense contractor whose intentions are always of the best sort. His gigs are of the usual action-hero variety — find stolen missiles, escape assassins, rescue civilians, humiliate dictators. But his near chastity, along with his staunch opposition to guns and capacity to solve every problem through the judicious application of chemistry and physics, sets him apart from other action stars. MacGyver is the thinking man’s hero.

Though, actually, when you go back to watch his adventures two decades after they first aired, you discover Mac’s target audience probably consisted mainly of boys, not men. I started watching the 139-episode DVD boxed set a few weeks ago, shortly after gadget blogs gleefully reported that Lee David Zlotoff, the series’ creator, said he was thinking of making a “MacGyver” movie. This jogged in me memories of boyhood, especially of how, after watching each MacGyver trick, I’d feel a bit invincible: I was small, but I was clever. Like MacGyver, I could take them.

But to adult eyes “MacGyver” is often too goofy by half. It’s not just that his tricks are improbable. At times — like when he interprets a deaf friend’s dreams to find clues to an impending missile theft — they seem to violate the show’s premise, that science beats brawn. In these instances, MacGyver doesn’t use science; he uses magic.

Then there are the children he befriends and the liberal orthodoxies he defends — tendencies that bump the show’s preachiness dial. Mac’s always popping up in foreign countries — Afghanistan, Myanmar — and running into kids and peasants who are oppressed by unsmiling overlords. In just about every second episode, he’s teaching kids about the dangers of guns, a position that, we learn in one episode, he came to as a boy, when a friend of his was killed by a gun. The antigun thing is a little specious, though: MacGyver’s got nothing but nothing but love for explosives, painful booby traps, fire extinguishers rigged up as projectiles, and enormous boulders that he sets up to fall on villains. The real reason he doesn’t use guns is obvious — he’d be able to shoot his way out of most traps, and that would be too easy.

I don’t mean to get down on “MacGyver.” There’s something in its flaws worthy of re-viewing, a particular moment in America preserved on TV. MacGyver is meant to exemplify a certain noble strain of American power. He doesn’t take the easy way out, and when in a jam, he uses what he finds around him to ingenious effect. If you strain you see a greater American story here too — that his ingenuity is frequently too good to be true, and leads to pat, uncomplicated endings that call for no greater reflection.

There’s also something striking about “MacGyver’s” moment in TV. Watch this show as a yardstick to measure how far we’ve come. Even the simplest dramas today — I’m looking at you, “CSI” — are complex and multilayered next to “MacGyver,” which underlines and explains everything, gums up all dialogue with exposition and introduces new, throwaway characters in each episode. There’s much hand-holding here: Even in foreign countries, everyone speaks English, every villain is one-dimensionally evil, and every tender moment is helped along by a swelling score.

But that’s why I hope someone makes a “MacGyver” movie. Mac needs a makeover. Lift him up to big-budget action standards — give him a story line that can span a couple of hours; give him a girl to love, but who may also cross him; give him a more complex mission (maybe to find out who’s putting all the salmonella in our salads?); and give the whole package fast, Paul Greengrass-style editing. Also, make sure one of his crazy solutions involves Mentos — people online go crazy for tricks with Mentos. Do all that and we might yet have a lasting American hero.

* * * * *

Read more of Salon’s Re-Viewed, offering a fresh look at great TV shows available on DVD.

Continue Reading Close

Goodbye to Machinist

Yo, I'm out.

  • more
    • All Share Services

Goodbye to Machinist

Today much of the tech world is sad that the iPhone 3G’s launch is going so miserably. But I’m sad that it’s my last day at Salon.

I’ve accepted a job at Slate, where, starting next week, I’ll be writing a twice-weekly technology column. Machinist will go on a break for a week, after which a guest blogger will bring you the latest tech dish.

I joined Salon in 2002, and since then I’ve written about, among other things, Smart Cars, robotic vacuum cleaners, muffin toasters, voting machines, architecture and 9/11, Tower Seven, Bill Clinton, Terri Schiavo, Florida’s elections, “The Wisdom of Crowds,” Malcolm Gladwell, Linux, Daniel Levitin, the copyright industries, Lawrence Lessig, The New York Times, Martha Stewart, a mnemonic to remember the Solar System’s planets, Google, garlic, stem cells, Comcast, Apple, Speedo, taxes, Social Security, Antonin Scalia, Barack Obama, the robots’ plan to take over the Earth, Howard Stern, Stringer Bell, Current TV, campaign finance reform, MoveOn, Howard Dean, Nintendo, Total Information Awareness, Java, Alice Waters, “The West Wing,” Robert F. Kennedy, Jr., the Washington Post, Judy Miller, and Hurricane Katrina.

I’m pretty sure there’s no other news outlet on the planet which would have given me such latitude, and I thank everyone here at Salon for that. I also want to thank all the readers who’ve read my work, not to mention praised me, hounded me, kept me up late at night swearing at the screen. Don’t ever change.

Continue Reading Close

“True Enough” at Google, and in San Francisco

A YouTubey presentation of my book.

  • more
    • All Share Services

As I mentioned in the comments yesterday, I’m getting ready to depart this space; I’ll have a fuller explanation tomorrow, sometime before or after I get in line to buy the new iPhone.

In the meantime, I thought I’d add a note about one of the more fun events related to my book’s release — the opportunity I had, in May, to speak at Google’s headquarters in Mountain View.

It was thrilling not only for the splendor of the place — even their commodes are computerized — and the welcoming attitude of my hosts at the Authors@ program (the company buys your books and hands them out to employees for free), but also because Googlers seemed to intuitively grasp my argument and posed many penetrating questions.

Google records these things and posts them up on YouTube, so if you’re looking for something to watch while eating a sandwich at your desk, have at it:

Another thing on the book: I’ll be reading and signing at Book Passage in the San Francisco Ferry Building next week — 6 p.m. on Thursday, July 17.

If you’d like to talk about facts, rumors, conspiracy theories, and spin in the digital age, do stop by.

Continue Reading Close

The iPhone 3G reviews are in: It’s pretty good

But battery life suffers, and the GPS isn't as great as you hoped.

  • more
    • All Share Services

Walt Mossberg (WSJ), David Pogue (NYT) and Edward Baig (USA Today) have been using the new iPhone 3G for a couple of weeks now, and today they all dish on their experiences.

They were all fans of the first model, and they’re pretty happy about the new model’s new features. The reviewers say the phone’s 3G network access leads to a much zippier Internet experience, that its audio quality has been dramatically improved, and that it cozies up to Microsoft’s corporate e-mail system.

But there are some drawbacks, too: Mossberg finds the battery life lacking, and Pogue says that that the phone’s GPS antenna is too puny to be of much use.

Here’s Mossberg on the battery life:

More important, in daily use, I found the battery indicator on the new 3G model slipping below 20 percent by early afternoon or midafternoon on some days, and it entirely ran out of juice on one day. I overcame this problem by learning to use Wi-Fi instead of 3G whenever possible, turning down the screen brightness and even turning off 3G altogether, which the phone permits.

The iPhone 3G’s battery life is comparable to, or better than, that of some other 3G competitors. But they have replaceable batteries. The iPhone doesn’t.

And Pogue on GPS:

Unfortunately, there’s not much you can do with the G.P.S. According to Apple, the iPhone’s G.P.S. antenna is much too small to emulate the turn-by-turn navigation of a G.P.S. unit for a vehicle, for example.

Instead, all it can do at this point is track your position as you drive along, representing you as a blue dot sliding along the roads of the map. Even then, the metal of a car or the buildings of Manhattan are often enough to block the iPhone’s view of the sky, leaving it just as confused as you are.

None of the reviewers were provided with applications that third-party developers are creating for the iPhone (these will go on sale at Apple’s online App Store). Mossberg, though, writes that he tried out some of these apps on an older phone, and was pleased with the results:

I tested a game that used the phone’s motion sensors to control the action, and I tested several programs from America Online (TWX), including AOL Instant Messenger; AOL Radio, which streams music from the Internet; and AOL’s Truveo video search engine. All worked very well.

These apps will also work on old iPhones as well as on the iPod Touch.

The iPhone 3G goes on sale Friday at 8 a.m. But some enterprising folks have gotten a hold of them already — check out the Boy Genius Report blog’s unboxing photos.

I talked about how to get an iPhone in my video for Current TV this week:

Continue Reading Close

Scary! YouTube ordered to hand your viewing history to Viacom

But there's a silver lining to one of the most bone-headed legal decisions in recent times.

  • more
    • All Share Services

Update: This post has been updated with comments from Viacom.

In the fall of 1987, a freelance reporter named Michael Dolan learned that judge Robert Bork kept an account at Potomac Video, a D.C. rental shop. This was at the height of the contentious and ultimately failed Senate confirmation hearings for Bork’s nomination to the Supreme Court — so naturally, Dolan thought there was a story here, and he went to work on getting a peek at Bork’s video rental history.

It wasn’t hard work. Dolan popped into Potomac Video one afternoon and asked if he could look at Bork’s movie file. “There sure are a lot of them,” the assistant manager replied. “Is it OK if I make a Xerox copy?”

That was OK with Dolan; weeks later, he published Bork’s rental history in the D.C. alt-weekly the Washington City Paper.

Bork’s taste in movies was itself unremarkable (“First off, despite what all you pervs were hoping, there’s not an X in the bunch, and hardly an R,” Dolan wrote). But the publication sparked outrage from groups on the right and the left — including the ACLU and People for the American Way, which had vehemently opposed Bork’s nomination.

In 1988, Congress, spurred by the fear that the press might now easily unearth all politicians’ movie habits, passed the Video Privacy Protection Act, which remains one of the strongest privacy laws in the nation. The law prohibits stores from disclosing video histories unless ordered to do so by a court — and even then, customers must be given “the opportunity to appear and contest the claim” of any party seeking to learn what you watched.

I tell you all this as a historical wind-up to yesterday’s shocking news: In the ongoing copyright battle between Viacom and Google, a judge ordered Google’s subsidiary YouTube to hand over an enormous trove of data identifying who watched what and when on the video-sharing site.

Viacom’s lawyers argued that they needed this data to prove that “infringing” videos — e.g., clips of “The Daily Show” and “The Colbert Report” — were more popular than non-infringing user-generated videos. Presumably, if it proves this, Viacom might prevail in its argument that YouTube’s bread-and-butter was illegal videos, and thus owes some of its success — and billions of dollars — to media companies.

The database in question is astonishingly broad: Viacom asked for 12 terabytes of logs (approximately 12,000 GB) that detail each instance in which someone pressed Play on a YouTube video, plus the YouTube username of the viewer who watched it, the date and time at which the user pressed Play, and the IP address of the viewer’s computer. The database covers videos seen both on YouTube as well as those embedded on other pages: If you’ve never visited YouTube but have clicked on a YouTube video from your daily newspaper’s Web site, you’re in the database.

Google objected to Viacom’s request on the grounds that producing the database would be expensive, time-consuming, and would invade YouTube users’ privacy. The judge — Louis Stanton of the Southern District of New York — Judge Stanton dismissed all Google’s arguments. The company’s “privacy concerns are speculative,” he wrote. (PDF here.)

Such pat reasoning should give you a general sense of the depth of bone-headedness in Stanton’s ruling. As Kurt Opsahl of the Electronic Frontier Foundation points out, the Robert Bork-inspired Video Privacy Protection Act applies not just to video cassettes but to “audio visual material” in general. Clearly it should apply here, and clearly, millions of YouTube users ought to have been given a chance to fight this invasion of our privacy.

But the real villain here isn’t Judge Stanton — it’s Viacom. I’ve previously raked the company over the coals for suing, rather than enjoying the fruits of, YouTube’s success (for instance last year, when it sacrificed potentially millions in ad dollars by pulling down the popular MTV clip of Britney Spears’ poor performance at the Video Music Awards).

But now Viacom’s sinking lower: Not content to fight just Google, the company looks to be manning the deck against us all. Sure, Judge Stanton might call this “speculative,” but think on it a bit: If Viacom’s willing to take on Google, what qualms will it have in suing you or me, recording industry-style, now that it knows what we did on YouTube? (Update: Viacom says it can’t use this data to sue you.)

All’s not lost. Google might manage to reverse this decision on appeal, and Viacom, gauging the outrage, could decide to withdraw or limit its request.

But our real hope here is legislative or regulatory action. Indeed, optimistic sorts might see a silver lining here.

As privacy scholar Jeffrey Rosen has written, “The politics of privacy tends to be largely reactive, fired by heartstring-tugging anecdotes that capture the public imagination.” Just as the airing of Robert Bork’s video history was the kick-start Congress needed to fix a clear privacy hole born out of then-new technology, this ruling might backfire on copyright holders, pushing lawmakers, finally, to curb the privacy-invading reach of copyright fights.

What we watch on YouTube is every bit as personal as what one rents from a store like Potomac Video. Indeed, it might be more private, and more salacious — imagine the fun you’d have if you were looking for unsavory data about a future Supreme Court nominee in 12 terabytes of YouTube logs!

In his floor speech in favor of the Video Privacy Protection Act, Vermont Sen. Pat Leahy argued that new database technologies capable of tracking private behavior called for new privacy regulations. That was two decades ago — and it remains true, still.

——

Update: A representative for Viacom e-mailed me to say that I’m overreacting. “We have no ability (and absolutely no desire) to use this data to sue end-users,” he argued, pointing out that all discovery documents in the case are bound by this confidentiality agreement.

Under this agreement, no one at Viacom will get to see these YouTube logs — only Viacom’s outside lawyers and experts, as well as court personnel, will have access to the data. The agreement also restricts the data to this case alone, which would seem to prevent the company from using the logs to sue users individually.

Michael Fricklas, Viacom’s counsel, told the New York Times, “I can unequivocally state that we will not use any of this information to enforce rights against end users.” He added that the company is looking into ways to “anonymize” the logs “to enhance the security of information that will be produced.”

Continue Reading Close

Page 1 of 143 in Farhad Manjoo