State Of The Internet/Surveillance Economy

Problem Statement

We have little real choice in our digital lives other than to acquiesce to systematic data collection and surveillance by corporations.* Digital experiences are optimized to extract as much data about ourselves, our environment, and our behavior as possible, while providing us with almost no opportunity to understand what data is being collected, who has access to this data, and how it’s being used to shape our behavior and opportunities. The surveillance economy gives the watchers “unprecedented ... power… distinguished by extreme concentrations of knowledge and no democratic oversight.” (Shoshana Zuboff). Yet these watchers are also sloppy and negligent, permitting their systems to be breached and gamed in ways that cause enormous harm to people, businesses, and society.

Information technologies—from the printing press to radio and television—have always created new social behaviors. But what’s different about today’s information technologies is that they don’t just broadcast to us, they interact with us. They monitor us in order to provoke specific responses—a click, a purchase, a feeling, a vote, an action. In other words, information technologies have gone from watching our behavior to shaping it. This behavioral manipulation is the engine of the Surveillance Economy, and data is its fuel.

“Google discovered that we’re less valuable than others’ bets on our future behavior. This changed everything.” - Shoshana Zuboff

The Surveillance Economy is about more than just privacy. It’s about power.

Privacy is, indeed, a way for users to regain some control. But privacy is just a salve, not a solve for the dynamics below the tip of the iceberg. With so many parties rallying around privacy today, we need to shine a light on the fact that people have been made powerless.

“Is it privacy we should care for? Or is it the feeling of powerlessness?” - Jess Mitchell, Researcher, Inclusive Design Research Centre, OCAD

“It's interesting how quickly people are willing to surrender power for convenience." - Douglas Rushkoff, Media Theorist and Author

“Making consent more effective is incommensurate with the scope of the problem.” - Ben Wizner, Director of Speech, Privacy & Technology Project, ACLU

The Surveillance Economy goes beyond the browser.

A “data rush” is attracting new players from environments beyond traditional technology. They’re making use of masses of data, innovating ‘smart’ tools to collect more. From smart cars to smart cities, from fitness trackers to toasters, the Internet is now far beyond the browser. These new technologies are Trojan Horses for always-on data collection and manipulation.

“To keep people safe online, we have to be outside of the browser.” - Peter Dolanjski, Director of Firefox Security & Privacy Products, Mozilla

“The older generation is thinking of the Internet as that thing that's on the computer. The younger generation is thinking of it as this stuff that's on my phone, it's in my purse, and then I'm walking around with it. The next generation is going to have it embedded in everything they do. God knows if there will be a browser, because you just move through the world. And in some ways, the more embedded it is in your environment, the less people perceive of it as surveillance.” - Douglas Rushkoff, Media Theorist and Author

The information asymmetry underpinning the Surveillance Economy disables collective action—by design.

There is a huge asymmetry of capacity and information between Google and 1 billion unique Google users. This leaves little space for collective sense-making, organizing, and action. Making matters worse, algorithms are continuously dismantling a sense of collective cohesion between people, further fractionalizing our power against the corporate hegemony.

“When you get this level of concentrated economic power, it very often leads to concentrated political power and these sort of very unhealthy marriages between state actors and corporations.” - Denise Hearn Author

"They know more about me than I know about them. It’s hard to have a fair or productive interaction with me when there is an extreme information asymmetry.” - Mozilla employee

Reactionary legislation has shaped one vision for the Information Economy: A Surveillance Economy.

Today, innovation is still happening faster than legislation can keep track of—which often means regulators find themselves reacting to situations, instead of proactively laying out the rules. From GDPR to CCPA, these policies are reacting to a scenario with marginal protections at best and unintended consequences at worst.

“The masters of the Surveillance Economy persuaded regulators early on, and still now, that innovation is the most important value and that it’s too early for regulation. This left the space unregulated and shaped one version of the Internet—a commercial one.” - Ben Wizner, Director of Speech, Privacy & Technology Project, ACLU

How can Mozilla find a path forward that takes advantage of the remarkable new insights ‘big data’ can bring while protecting individuals as well as the public?

*although corporate and government surveillance are intertwined, for this project -- and at this stage -- we looked only at the corporate aspect and its impact on people.

Research + Community Input

To explore different dimensions of the Surveillance Economy, we started from comments and thoughts shared by employees and community members through employee workshops and MozFest.

We then spoke to over 45 people* representing a wide range of perspectives on this space — from policy wonks, economic theorists, and heads of government to Mozilla product managers, community members, Mozilla Fellows, and diversity, equity, and inclusion activists. We asked each to help us understand the State of the Surveillance Economy: What is it? How did we get here? Why is it so insidious? How, exactly, does it hurt people and companies? And, most importantly, what can we do about it?

Working from some of our early findings, we created three design provocations and shared them with Mozilla community members and employees for voting and feedback. Mozilla has a remarkable asset in our diverse communities, so including their voices in this conversation was critical to our learning. These crazy departures and directions gave us more insight into what directions resonated most, and helped to bring in new thinking too.

In parallel, we ran traditional -- but limited -- market research, built a graph and clustered topics from a Wikipedia crawl to help ensure subject breadth, and generally immersed ourselves in the world of surveillance.

Finally, we incorporated employee and community feedback from All Hands into the final suggested Explorations.

* We did not speak to many 'end users' as part of our inquiry, due to scope and timing limitations. However, the human voice will be critical for the next phase around each Exploration.

The Problems

Through discussion, research and workshops, we distilled three big problems that we felt lie at the heart of the Surveillance Economy:

• Exposed: The burden of protection is put on people—who can’t see or feel the threat.
• Excluded: Big companies are hoarding troves of data and smothering competition.
• Exploited: The only thing not surveilled is how people’s data is used.

People are Exposed in the Surveillance Economy

• Opting out is not an option

To engage with the Internet, you have to be willing to give up data. It’s all in or all out, with little control over what is shared in different contexts.

• Privacy is not user friendly

It’s clunky and requires continuous time, attention and knowledge to set up and maintain.

This is perpetuated by:

Corporations don’t see themselves as responsible The ideology of Silicon Valley is that technology is agnostic, and so it—and the people that make it—are not accountable to design with people’s privacy in mind.

Always more, and always on While people have low visibility of corporate surveillance on their lives, corporations are striving to have ever more visibility of people: more touch points, more environments, and more data.

Little power to the people People have become atomized customers, stripped of any potential power for collectives to demand new protections from corporations.

A mismatched market People want convenience. Corporations want surplus. Companies supply the convenience people demand, extracting as much data as possible in return. It’s not a fair or transparent exchange.

“As designers we aim for ease-of-use and don't think about opt-in/out.” - Caroline Sinders, Mozilla Community Member

“People really don't know how to solve this—it's like the tsunami has already started, and we can't roll back the wave.” - Denise Hearn, Author, The Myth of Capitalism

“Right now, the choice is binary; opt in or opt out. We need more tools, practices, products, to empower people to take more action.” - Katharina Borchert, Chief Innovation Officer, Mozilla

“Don't ask me if I want a cookie when I'm reaching for the cookie jar.” - Cory Doctorow, Sci-Fi Author, Special Advisor, EFF

“It's interesting how quickly people are willing to surrender power for convenience… Most algorithms are there to divide us, because an atomized person is a better customer.” - Douglas Rushkoff, Media Theorist and Author

“There has been a violation of fiduciary structure. Lawyers and doctors have it —when you have privileged information, there is an asymmetry of power, and the law says these people can’t manipulate people based on it to buy services from you they don’t need, for example. Companies shouldn’t be able to collect data, which is the privileged information of the modern era, without a fiduciary responsibility for its use.” - Daniel Schmachtenberger, Civilization Theorist, Center for Humane Technology

“Google doesn't need to spy on you—you're intentionally giving them your data.” - Ekr Rescorla, CTO Firefox

“The way we design user interfaces can have a profound impact on the privacy of a user’s data. It should be easy for users to make choices that protect their data privacy. But all too often, big tech companies instead design their products to manipulate users into surrendering their data privacy. These methods are often called ‘Dark Patterns.’” - Alexis Hancock, Staff Technologist, EFF

Towards a Positive Vision

At an early employee workshop in Berlin, someone commented that we lack a positive vision for a 'datafied' society. The world needs a vision for a healthier alternative to the Surveillance Economy --- and they want Mozilla to help define and create a more favorable future.

“We need to organize the Internet around new values...Mozilla has these values: privacy, civil society, user control, along with a creative and active legal team.” - Ben Wizner, Director of Speech, Privacy & Technology Project, ACLU

Further Reading

There is a lot of fantastic documentation and analysis of the surveillance economy....

• Shoshana Zuboff, The Age of Surveillance Capitalism (book) + our IRL podcast.
• Kashmir Hill, ”I Got Access to My Secret Consumer Score. Now You Can Get Yours Too” (paywall)
• New York Times' series The Privacy Project
• Mitchell Baker’s interview with Cheddar about rebuilding consumer trust in the internet