Mozilla Network Outages Data Project

From MozillaWiki
Jump to: navigation, search

Introduction

The Network Outages Data Project was initiated at Mozilla in 2019 to explore mission-aligned innovation on use of Mozilla data to support a healthier internet worldwide, by:

  • (a) creating a dataset of signals based on existing Mozilla telemetry that correlate with network outages in countries and cities worldwide.
  • (b) validating the reliability of outages signals with the help of internal network experts and renowned external experts.
  • (c) disseminating our findings and publicly releasing the dataset.

Project Background

Validation

In order to validate the data set prior to its release, we signed legal agreements with the following organizations prior to sharing it with them and hosted a series of group meetings (starting July 2020) to discuss their findings and feedback. All data under the data set was (and is) anonymized and aggregated to ensure user privacy, with no data included for any unknown location or any city with a population less than 15,000 people. All included locations also had at least 50 active users in a given hour.

Mozilla members of the Internet Outages Data (IOD) working group then conducted individual interviews over Zoom with researchers from these organizations:

  • The Open Observatory of Network Interference (OONI)
  • RIPE NCC / RIPE ATLAS
  • Measurement Lab (M-Lab)
  • The Internet Outage Detection and Analysis (IODA) of Center for Applied Internet Data Analysis (CAIDA) at the San Diego Supercomputer Center
  • Internews
  • Access Now #KeepitOn Coalition

The interviews were conducted over the course of June and July 2021, by at least two members of the relevant Mozilla working group, with one technical member from the Data team always present. Each interview lasted 45 minutes and was followed by a 10-15 minute debrief session for the Mozilla team to revise notes, reflect on learnings, and add any potential action items.

We directed all participating organizations to compare and contrast data from the following countries and documented internet outages with very different characteristics.

  • Belarus in August 2020
  • Uganda in January 2021
  • Myanmar in February 2021

Validation Outcome

Researchers from all six organisations unequivocally confirmed that they found Mozilla’s dataset useful for detecting internet outages. They validated the insights gained from Mozilla’s data by matching them against their own sources as well as from other publicly available data (such as the IODA repository). There was a broad but clear correlation between their sources and Mozilla’s data.


Advantages
Some of the advantages of Mozilla dataset that were identified by the organizations were:
  • corroborative and cross-validation value
  • diversity and breadth of clients (sending data)
  • ability to see the nuance of aggregated user behaviour and internet outages
Disadvantages
Some of the disadvantages of Mozilla dataset that were identified by the organizations were:
  • Lacking granularity (aggregated pings v/s actual information)
  • Inability to segregate data at a non city-level (and other governance districts)
  • Pings do not record timestamps of when the data was recorded and when it
  • attempted to be sent (inc. multiple times), making it harder to detect smaller
  • shutdowns.

Contact

The team can be contacted on outages@mozilla.com

Team leaders
Alessio Placitelli, Mozilla Data Organization
Udbhav Tiwari, Mozilla Policy
Solana Larsen, Mozilla Foundation Insights