SALON

Saturday, Jun 15, 2013 3:00 PM UTC

PRISM software works just like Facebook ads

Data-mining expert Gregory Piatetsky-Shapiro says the spy technology has been around for years

By

Topics: International Business Times, Prism, technology, Facebook, NSA, data mining, , , , , ,

PRISM software works just like Facebook ads (Credit: Shutterstock)
This article originally appeared on International Business Times.

International Business Times

The National Security Agency has been involved in intelligence-gathering schemes since its inception in 1949, but 21st century technology has advanced far beyond the wiretap and the codebook. Modern intelligence gathering, like the recently unveiled PRISM program, is the product of the “big data” era.

“There’s nothing surprising technically” about programs like PRISM, data-mining expert and KDNuggets editor Gregory Piatetsky-Shapiro said in a phone interview.

Most technology experts can’t speak with too much certainty about a program they haven’t seen, like PRISM, but based on publicly available information, something like PRISM has been possible for years. The same innovations in software and hardware that aid your Google query or help advertisers track your habits online — like when you examine a book on Amazon and then see an ad for that book pop up later on Facebook — also allow the NSA to sort through reportedly tens of billions of pieces of information a month.

One of the major components of PRISM is believed to be an open-source database called Apache Accumulo, which the NSA began working on in late 2007. Originally called CloudBase, Accumulo is built on top of a software framework called Apache Hadoop and is similar to Google’s BigTable storage system. (If you would like to buy Accumulo for yourself, some of the developers that worked on the project with the NSA sell a commercial version through their company.

“Accumulo’s ability to handle data in a variety of formats — a characteristic called ‘schemaless’ in database jargon — means the NSA can store data from numerous sources all within the database and add new analytic capabilities in days or even hours,” Derrick Harris wrote for GigaOM.

Some of the other advances aren’t necessarily of the hardware or software variety. The science of studying networks has been growing by leaps and bounds, allowing analysts to tease relationships from seemingly unrelated data points.

“If the NSA just has the metadata — who calls whom — that’s sufficient to determine the status of people,” Piatetsky-Shapiro said. “You don’t necessarily need the conversation if you have the network.”

Piatetsky-Shapiro pointed to a humorous Slate article published Monday that imagined British agents flagging Paul Revere as a person of interest based on his relationships with other colonial independence agitators.

“Rest assured that we only collected metadata on these people, and no actual conversations were recorded or meetings transcribed,” Duke University sociologist Kieran Healy wrote for Slate. “All I know is whether someone was a member of an organization or not. Surely this is but a small encroachment on the freedom of the Crown’s subjects.”

The leaks have also highlighted the degree to which almost all Internet communication is part of a giant interconnected and tangled web. For example, online, the difference between a foreign communication — which the NSA might flag, and a domestic one, which it shouldn’t, can sometimes get a little hazy.

“One interesting tidbit from the Guardian leaks is how much the U.S. is the center of global communications,” Piatetsky-Shapiro said. Internet communications often “take the cheapest route.” So, because the U.S. has so much available capacity, an email from, say, Pakistan to Canada, could be routed through America.

When one imagines how the NSA might go about analyzing the content from all the different types of communications it has stored, a line of spooks in a darkened room poring over emails is not what comes to mind these days. The customer service industry is already using language-processing algorithms to break down spoken and written sentences. By breaking down sentence structures and weighing the words contained in any of the various modes of communication, it’s possible for a program to organize messages by “intent.” It’s not too big a leap to assume that whatever actual conversations the NSA does capture would be sifted by a similar algorithm.

“If that stuff’s being used in the commercial world, it’s logical to assume they’d be using it as well,” Forrester Research analyst Glenn O’Donnell said in a phone interview.

More International Business Times

Roxanne Palmer

More Roxanne Palmer.

You Might Also Like

More Related Stories

Featured Slide Shows

7 motorist-friendly camping sites

close X
  • Share on Twitter
  • Share on Facebook
  • Thumbnails
  • Fullscreen
  • 1 of 9

Sponsored Post

  • White River National Forest via Lower Crystal Lake, Colorado
    For those OK with the mainstream, White River Forest welcomes more than 10 million visitors a year, making it the most-visited recreation forest in the nation. But don’t hate it for being beautiful; it’s got substance, too. The forest boasts 8 wilderness areas, 2,500 miles of trail, 1,900 miles of winding service system roads, and 12 ski resorts (should your snow shredders fit the trunk space). If ice isn’t your thing: take the tire-friendly Flat Tops Trail Scenic Byway — 82 miles connecting the towns of Meeker and Yampa, half of which is unpaved for you road rebels.
    fs.usda.gov/whiteriveryou


    Image credit: Getty

  • Chattahoochee-Oconee National Forest via Noontootla Creek, Georgia
    Boasting 10 wildernesses, 430 miles of trail and 1,367 miles of trout-filled stream, this Georgia forest is hailed as a camper’s paradise. Try driving the Ridge and Valley Scenic Byway, which saw Civil War battles fought. If the tall peaks make your engine tremble, opt for the relatively flat Oconee National Forest, which offers smaller hills and an easy trail to the ghost town of Scull Shoals. Scaredy-cats can opt for John’s Mountain Overlook, which leads to twin waterfalls for the sensitive sightseer in you.
    fs.usda.gov/conf


    Image credit: flickr/chattoconeenf

  • Nordhouse Dunes Wilderness Area via Green Road, Michigan
    The only national forest in Lower Michigan, the Huron-Mainstee spans nearly 1 million acres of public land. Outside the requisite lush habitat for fish and wildlife on display, the Nordhouse Dunes Wilderness Area is among the biggest hooks for visitors: offering beach camping with shores pounded by big, cerulean surf. Splash in some rum and you just might think you were in the Caribbean.
    fs.usda.gov/hmnf


    Image credit: umich.edu

  • Canaan Mountain via Backcountry Canaan Loop Road, West Virginia
    A favorite hailed by outdoorsman and author Johnny Molloy as some of the best high-country car camping sites anywhere in the country, you don’t have to go far to get away. Travel 20 miles west of Dolly Sods (among the busiest in the East) to find the Canaan Backcountry (for more quiet and peace). Those willing to leave the car for a bit and foot it would be remiss to neglect day-hiking the White Rim Rocks, Table Rock Overlook, or the rim at Blackwater River Gorge.
    fs.usda.gov/mnf


    Image credit: Getty

  • Mt. Rogers NRA via Hurricane Creek Road, North Carolina
    Most know it as the highest country they’ll see from North Carolina to New Hampshire. What they may not know? Car campers can get the same grand experience for less hassle. Drop the 50-pound backpacks and take the highway to the high country by stopping anywhere on the twisting (hence the name) Hurricane Road for access to a 15-mile loop that boasts the best of the grassy balds. It’s the road less travelled, and the high one, at that.
    fs.usda.gov/gwj


    Image credit: wikipedia.org

  • Long Key State Park via the Overseas Highway, Florida
    Hiking can get old; sometimes you’d rather paddle. For a weekend getaway of the coastal variety and quieter version of the Florida Keys that’s no less luxe, stick your head in the sand (and ocean, if snorkeling’s your thing) at any of Long Key’s 60 sites. Canoes and kayaks are aplenty, as are the hot showers and electric power source amenities. Think of it as the getaway from the typical getaway.
    floridastateparks.org/longkey/default.cfm


    Image credit: floridastateparks.org

  • Grand Canyon National Park via Crazy Jug Point, Arizona
    You didn’t think we’d neglect one of the world’s most famous national parks, did you? Nor would we dare lead you astray with one of the busiest parts of the park. With the Colorado River still within view of this cliff-edge site, Crazy Jug is a carside camper’s refuge from the troops of tourists. Find easy access to the Bill Hall Trail less than a mile from camp, and descend to get a peek at the volcanic Mt. Trumbull. (Fear not: It’s about as active as your typical lazy Sunday in front of the tube, if not more peaceful.)
    fs.usda.gov/kaibab


    Image credit: flickr/Irish Typepad

  • As the go-to (weekend) getaway car for fiscally conscious field trips with friends, the 2013 MINI Convertible is your campground racer of choice, allowing you and up to three of your co-pilots to take in all the beauty of nature high and low. And with a fuel efficiency that won’t leave you in the latter, you won’t have to worry about being left stranded (or awkwardly asking to go halfsies on gas expenses).


    Image credit: miniusa.com

  • Recent Slide Shows

  • Share on Twitter
  • Share on Facebook
  • Thumbnails
  • Fullscreen
  • 1 of 9

Related Videos

Comments

 2 Comments

Comment Preview

Your name will appear as username ( settings | log out )

You may use these HTML tags and attributes: <a href=""> <b> <em> <strong> <i> <blockquote>

follow salon

Most Read

Popular on Reddit

links from salon.com

vote
Let's Stop Screwing Around and Impeach James Clapper: After lying to Congress about all the different ways the NSA's spying on us, Director of National Intelligence James Clapper lies about how he lied. Call the White House at 202 456-1111 and Tell Obama to Get Rid of Clapper
128 points129 points | 9 comments
vote
Machiavelli doesn’t belong to the 1 percent | "The Prince" is oft-quoted on Wall Street, but its author was a hero of the working class who despised elites
69 points70 points | 19 comments
vote
We must hate our children
63 points64 points | 11 comments
vote
Machiavelli doesn’t belong to the 1 percent | "The Prince" is oft-quoted on Wall Street, but its author was a hero of the working class who despised elites
53 points54 points | 16 comments
vote
Let's Stop Screwing Around and Impeach James Clapper: After lying to Congress about all the different ways the NSA's spying on us, Director of National Intelligence James Clapper lies about how he lied. Call the White House at 202 456-1111 and Tell Obama to Get Rid of Clapper
63 points64 points | 6 comments

From Around the Web

Presented by Scribol
listener_new_rect The Listener: "The Joker": Laughing at life's deepest sorrows