The world of data has its own economics. If you know one thing about one person, you don’t have much. If you know one thing about nearly everyone or nearly everything about one person, you have a little. But if you know nearly everything about nearly everyone, you’ve got something priceless. Essentially, data giants are middlemen who connect buyers with sellers for a fee. Google, for example, takes a place among the premier content providers in the world. Every day, the company handles millions of searches for its users. But mainly, it creates lots and lots of lists. Google became what it is because its lists are very useful to millions of users. But in nearly every case, what a user wants is not provided by Google itself. Google just connects what the user wants with a list of relevant web pages. Google’s famous web crawlers search the Internet, making lists and rendering those lists to users.
All companies face growing competition online, where a local business is no nearer than a competitor on the other side of the world: just one click away. When ordinary companies add more computing power, it’s a necessary expense, something required to make and sell their oﬀerings. But for the data giants, each new data center is an end in itself and a competitive weapon. Traditional companies accumulate information about a narrow range of activity. Companies record who their customers are, where they are located, what and how much they buy, and the prices they pay. In the past, companies ascertained rudimentary facts about their customers by observing, by asking questions, and by using data brokers. But they lacked the data, the access, and the analytical resources to assemble a granular picture of customers.
The data giants are fundamentally diﬀerent. Companies like Amazon or Facebook know (or infer) not just who you are but what you are like. They know not only where you are but they can guess where you are going. They don’t just know what you are doing right now—they have a pretty good idea why you are doing it. And they make excellent guesses about what you will do next, guesses that grow more accurate every day as you go about the business of daily life while being carefully observed by the data giants.
For the data giants, the Internet isn’t an abstraction, and it certainly isn’t a utopian space where all are treated fairly. The Internet is a loose collection of physical equipment owned by competing groups. It’s a commercial battleground where a few companies dominate the ﬁeld. The most important bits of the Internet are in the buildings like Amazon’s facility in Ashburn that look like big-box stores.
What consumers don’t realize is this: They are in those structures. The most detailed report prepared by analysts working for the Stasi or the KGB (or our very own CIA for that matter!) doesn’t begin to compare with the comprehensive data wake shed by each consumer. Every minute of the day we shed data in profusion. Every movement, gesture, word, and keystroke creates additional data. Computers, tablets, cell phones, and sensors all around us pick up huge and ever-growing quantities of intimate information, then record, tabulate, and analyze it. For the privileged few with the access and ability to read it, a data wake shows what happened, why it happened, and increasingly what will happen next.
Today’s data giants vie for what is likely to be the richest commercial prize in history. The history of technology is full of great enterprises. More than a century ago, the Industrial Revolution spawned giant enterprises: companies like Standard Oil, U.S. Steel, and American Tobacco, which dominated whole industries. The giant trusts delivered unprecedented proﬁts and built individual fortunes never before possible. The Ten are positioning themselves to dominate not merely a single industry but many industries at once, perhaps even entire economies.
That’s because data giants are positioned to act as middlemen on virtually every transaction, matching buyers with sellers while getting paid for that role. Gradually, the data giants are already capturing a growing share of the economy’s surplus. Knowing the customer intimately allows them to set various prices quickly and accurately, tailoring prices to individual need and the ability to pay from moment to moment. The more you need an item and the fewer your options, the more you pay. Engineering each transaction applies not only to pricing. Given suﬃcient data, quality and service can be individually tailored as well. A data giant can ascertain the minimum quality and service each customer will accept. Capturing a growing portion of the consumer surplus by charging more for almost everything represents an enormous commercial prize, a fortune bigger than has ever previously been possible.
The producer surplus
The massive collection of data doesn’t only aﬀect the consumer surplus. Companies also risk losing their proﬁt margins, the mirror image of the consumer surplus. Virtually all businesses sell their oﬀerings for some degree of proﬁt. The amount of proﬁt on each sale varies depending on many factors. Just as in the case of consumers willing to pay something more for their purchases if they had to, companies would be willing to take a lower price if they had to. Whenever a company makes a sale, there is necessarily an acceptable lower price that still delivers a proﬁt until the point of indiﬀerence, when the company doesn’t care whether a given sale occurs. Added up across the entire economy, the total proﬁt made by sellers represents the producer surplus.
Like the consumer surplus, the aggregate producer surplus is a very large quantity. Corporate proﬁts as a whole add up to something in the neighborhood of $2 trillion a year, and proﬁts to the owners of unincorporated businesses add more than $1 trillion more. Businesses don’t give up their proﬁts without a ﬁght, but the data giants have already started nibbling away at this more than $3 trillion prize.
Individual companies tend to be very reluctant to reveal how much they make on each sale, referred to as their “margin.” It’s an accepted principle of business that a buyer who knows the seller’s margin can demand a better price, squeezing the margin and cutting into proﬁts. In theory, if the buyer with suﬃcient clout knows the details, he or she can take all, or nearly all, of a seller’s margin.
In the past, ﬁguring out a seller’s margin wasn’t easy. Sellers guard their margins as trade secrets. But as with consumers, big data is gradually changing the balance of power for sellers. Given suﬃcient information, an astute buyer can deduce the seller’s margin despite the seller’s eﬀorts to conceal it. Walmart, to take one example, has made a science of studying its supply chain and squeezing supplier margins at every opportunity. Walmart’s extensive knowledge of not only its current suppliers but all other alternatives in the marketplace has allowed the company to bargain from a powerful position. The suppliers can’t pretend they are oﬀering their best prices when Walmart knows full well they are still making money. Walmart scientiﬁcally studies its suppliers’ proﬁts and very accurately judges about how much it has to pay. And given Walmart’s giant scale, few sellers can just refuse to do business with it.
But even Walmart isn’t a true data giant. While it has an impressive capability and scale, the company lacks a truly massive data funnel that would allow the pervasive collection of data in real time across the economy. The data giants are positioned to study companies as well as individuals, and the data wake left by companies is a rich trove. Historically, companies have protected themselves by operating their own data pools and requiring conﬁdentiality agreements with workers. Over time, those protections will erode as the ability to maintain the secrecy of individual facts becomes a thing of the past. Secret formulas and manufacturing processes can be photographed with cellphone cameras, stored on miniature memory devices, and transmitted instantly around the world or published for all to see. Eventually, individual secrets will all but cease to exist, and the business paradigm will shift from creating value out of particular information to creating value from very large bodies of information. As data giants continue to vacuum up granular information in real time, they create an asset so vast that it can’t be stolen or copied. Not because it’s protected, but because it’s so big.
Look at wages, for example. For a data giant, identifying all of the workers at a given company would be child’s play. Location data from phones, cameras, and road sensors readily reveal who comes to the oﬃce. E-mail, web, phone, and travel records ﬁll out the rest of the picture. Other sources ﬁll out the details: roughly how much is earned and who is hired or ﬁred, all in real time. The data giants can eﬀectively study a company’s workforce and fairly easily draw conclusions about a company’s costs structure.
Intellectual property is another area with falling barriers. Historically, companies have gone to great lengths to protect their secret formulas and methods. But the falling cost of collecting and storing data leads to increasing opportunities for secrecy breaches. Imagine a manufacturing plant with a proprietary manufacturing process. Even if the intent is entirely innocent, over time, a set of pictures taken inside the plant will collect on Facebook and other photo-sharing sites. Employee birthday photos, bulletin-board pictures, and other seemingly innocuous items contain background clues that can be teased out via big data. With all the attention and patience of an archaeologist, but operating at lightspeed, the vast electronic brains of a data giant could, in theory, assemble a mosaic revealing the machinery and layout inside a manufacturing plant. With all their resources and capabilities, substantial companies are considerably less vulnerable than consumers to the predations of a data giant. But eventually, the massive collection of data will overwhelm all eﬀorts at secrecy, putting even the largest companies at risk.
Individual secrets—a formula, a process, a price—will readily be discerned through the collection and analysis of data. In the end, the only remaining secrets will be the huge troves of data in the hands of the data giants, assets too massive to copy or exchange. No one but a data giant will control the resources to understand and use the treasures that lie within.
The data giants have begun down a path that leads to almost god-like omniscience. There is now a foreseeable prospect of knowing what every individual will pay for every single thing at each moment in time, as well as knowing what every seller or manufacturer will charge. The diﬀerence between the most an individual will pay and the least a seller will accept, the “surplus” of the entire economy, is the potential reward. With suﬃcient data and analytic power, a data giant is well positioned to charge for matching a supplier to a consumer. At the maximum, the data giant can absorb much or all of the entire surplus so that the consumer pays all he or she can pay, and the producer ends up with the least it will accept.
The Third Surplus: Labor
Along with the consumer surplus and the producer surplus, labor represents a third target for big data. Like any other sellers, workers sell their skills and eﬀorts. The labor marketplace has historically been among the most ineﬃcient areas of the economy, with large costs falling on companies seeking to identify and hire employees. Workers also face their own diﬃculties in identifying appropriate opportunities and negotiating pay. These widespread ineﬃciencies give rise to excess unemployment, labor unions, and myriad delays and costs that together constitute a signiﬁcant drag on the labor market.
Big data has begun to fundamentally change the relationship between workers and employers. With the massive collection of granular employee data, companies can increasingly micromanage the output and productivity of each individual. Eventually, the crude science behind time-and-motion studies will be reﬁned to more scientiﬁcally measure achievements in the workplace.
Workers are in a potentially precarious position. The tremendous information imbalance between the employer and the worker makes it diﬃcult for an individual to ﬁgure out the value of his or her contribution. What’s more, companies tend to be secretive about what they pay each worker. Individuals have limited means of assessing the “market” rates for their jobs.
Companies already harness the power of data to increase their bargaining power with workers. Electronic workplace surveillance, e-mail and phone records, and web-surﬁng histories provide a trove of data useful not only for weeding out bad apples but also for inﬂuencing desired behavior and negotiating pay. Companies have more opportunities than ever before to assess who is looking for a new job and who is not, who might leave a job, and who will stay no matter what. And companies have a whole range of opportunities to modify behavior. Policies against drugs, alcohol, smoking, and overeating—even outside the workplace—can now be backed up with intrusive surveillance and data collection.
Take the case of a single mother in St. Louis employed as an oﬃce worker until ﬁred for publishing an anonymous blog salaciously detailing her amorous escapades. Despite taking care to keep her work life separate from her private life and blog, she was unable to fully hide her identity online. Thanks to the total recall of Twitter’s search engine Topsy, her supervisor discovered the connection between her real name and her provocative blog, leading to her dismissal. Whether or not one approves of racy blogging, her story is a cautionary tale about the perils big data holds for employees. Few legal protections exist for employees when employers disapprove of oﬀ-duty behavior, and whatever protections do exist tend to be narrowly interpreted.
As massive data shifts the power balance from workers to employers, it shifts even more power to the data giants. Your company may be able to detect if you have been in e-mail contact with a recruiter, but Microsoft or Google or Verizon knows whether you have an interview, and when. The data giants are well positioned to sell their services to companies seeking to manage their workforce. Taken to the extreme, a data giant could be able to deﬁne the terms of employment across large populations.
The Power of the Middleman
Amazon provides a good example of the enormous economic potential for data giants. Because of its scale and its accumulated knowledge of suppliers, the company is well positioned to bargain for cost eﬃciencies and lower margins from its suppliers. Amazon’s ownership of important data infrastructure, the server farms of AWS, for example, gives the company a huge window on the economy through which it can obtain granular data from millions of sources.
If Amazon knows it can charge an individual a bit more for a product, the company doesn’t have to share that extra money with a supplier. Amazon can just keep it. Today the company’s proﬁts are not very impressive compared to its sales. Retailing is competitive, and Amazon has not, at least so far, ﬁgured out the magic formula to make big proﬁts while driving competitors out of business. But the pressure Amazon places on competitors is intense, and the future proﬁt potential is tantalizing. That’s a plausible explanation of why Amazon’s stock trades at a high value despite its meager proﬁts. If Amazon gains the upper hand in retailing, proﬁts will follow in due course.
Amazon’s rapid rise to the top tier of retailing has given the company substantial market power. In book publishing, for example, something in the neighborhood of one-third of the market belongs to Amazon. So when the company reached an impasse with the large publisher Hachette, it simply stopped selling some of the titles published by Hachette imprints. Determined purchasers still found the books from other sellers, but the cost—and message—to Hachette was clear.
Amazon’s retail network and big-data capabilities are easy to understand. It’s a little less obvious how Facebook and Google are becoming powerful middlemen on billions of transactions. After all, Google and Facebook aren’t retailers. But they are advertisers. When you click through an ad from a data giant and make a purchase, the data giant gets its cut. How much Google or Facebook gets is not apparent to the customer because that happens behind the scenes. But it would not be diﬃcult for any of the data giants to negotiate as follows: When the consumer clicks on an ad for toothpaste, the data giant knows who he or she is. The ﬁeld is currently in its infancy, but before long the data giant will be able to assess what the consumer is willing to pay. In a microsecond or two following that ﬁrst click, Facebook can convey a message to the toothpaste seller: “I have a customer interested in your product who will pay $5.00 for it.” Because Facebook can capture very detailed knowledge of the seller’s cost structure, in the next millisecond Facebook reveals the catch: “The Facebook commission will be $2.50 for this customer. Would you like to accept this proposal?”
The building blocks of market dominance are already in place. In his book The Search, John Battelle tells the story of an online shoe retailer, 2bigfeet.com, which sells large sizes of men’s shoes. Early in the company’s history, the website’s popularity naturally put 2bigfeet.com near the top of Google searches for oversized shoes. In 2004, without warning or explanation, Google changed its search algorithm. As a consequence, 2bigfeet.com all but disappeared from the Internet, demoted from the ﬁrst page to the ﬁftieth page. After the company’s business nearly dried up, it didn’t take long for 2bigfeet.com to ﬁgure out a solution. The shoe seller began buying ads from Google and shot right back to the top of the results, where it remains today. The 2bigfeet.com story is a neat illustration of how powerful Google’s business model is. The company’s original search technology gained popularity and users. User growth increased Google’s scale, providing the company lots of data to improve its search technology. Over time, Google outpaced all of its competitors in search, and now the company’s immense scale makes it almost impossible for most companies to avoid. Like it or not, if a business wants to be found online, Google is well positioned to charge a toll. Nobody is forced to pay, of course. Any business that is comfortable with not appearing as a search result can completely ignore Google. But there aren’t many companies left that can aﬀord to be invisible to online search. Meanwhile, the data giants are creating a new class system in the economy. At the top are organizations, like Google or Facebook, that have the data and machines to orchestrate vast portions of the economy, gobbling up surplus and extracting value on a scale never before imagined. Below them are lesser companies and businesses that carve out speciﬁc niches, perhaps holding a patent or (for now) a secret edge that can’t be readily duplicated. And at the bottom will be a huge number of businesses and consumers increasingly disadvantaged by their complete inabilities to contend with the data giants on equal terms.
Excerpted from "All You Can Pay: How Companies Use Our Data to Empty Our Wallets" by Anna Bernasek and D.T. Mongan. Published by Nation Books. Copyright 2015 by Anna Bernasek and D.T. Mongan. Reprinted with permission of the publisher. All rights reserved.