Security/FirefoxOperations

From MozillaWiki
< Security
Revision as of 17:18, 10 February 2017 by Ulfr (talk | contribs)
Jump to navigation Jump to search

Firefox Services & Operations Security

The FoxSec team is tasked with securing core Firefox services operated by the Firefox Services Engineering and Operations organization at Mozilla.

Foxsec1024.png

Contact

Email us at foxsec@mozilla.com.

To report a security issue on a given site, use the bug bounty form as explained here.

Backlog

The table below summarizes the open issues assigned to the FoxSec team, sorted by area of focus.

Operational Security

Continuous Testing (TDS)

Fraud Detection

User management

Infra Hardening

Threat monitoring

4 MEDIUM
5 LOW

2 HIGH
3 MEDIUM
5 LOW

1 HIGH
2 MEDIUM

7 MEDIUM
8 LOW

1 LOW

Application Security

Risk & Security reviews

Test & Implement Baseline Security

Data & Code Signing

Training & Communication

Bug Bounty

External audits

3 MEDIUM
3 LOW

2 HIGH
8 MEDIUM
6 LOW

2 HIGH
1 MEDIUM
1 LOW

1 MEDIUM
6 LOW

2 LOW

1 HIGH
1 MEDIUM
1 LOW

Strategy

1. Improve operational security of the core infrastructure

1.1 Implement Test Driven Security (TDS) in CI/CD

Security tests should be part of the continuous integration (CI) and continuous delivery (CD) pipelines.

  • CI integration should be part of the code commit/review process, either in an existing CI (travis-ci, circleci, taskcluster) or in a security dedicated one. CI tests should include static code analysis and recommendations, docker containers testing and dependency checks (vulnerability management).
  • CD integration should be done at Jenkins' level, when stage environments are built and promoted.All services are regularly rebuilt by Jenkins. CD tests should include application vulnerability scanning (ZAProxy) and infrastructure access control tests (security groups, IAM permissions, ...).

TDS should output directly in the build pipeline at first, and allow dev & ops to control levels that block integration & delivery. In a second phase, TDS outputs should be aggregated into a central security tracking platform.

1.2 Make use of the logging pipeline to detect fraud and anomalies

Heka, ElasticSearch and Kafka are powerful tools on top of which we can plug various pattern detection mechanisms to identify known bad actors, or unusual behavior. Fraud detection is a highly requested feature that devs don’t want to rebuild every time. Fraud detection should operate autonomously for each service, taking into account business rules set by the developers and the security team.

1.3 Improve user management and authentication

We should make better use of LDAP to add and remove employees from various third party services and admin panels.

  • Admin panels should rely on Mozilla's Identity Management platform provided by IT
  • Third-party services (datadog, pagerduty, aws) should have automated user management (userplex).

foxsec need to facilitate integration with Mozilla's IAM with standard libraries and tools.

1.4 Harden the infrastructure

All services and tools that are part of the standard infrastructure should undergo security hardening. Hardening rules should be testable in the CD pipeline (see TDS above) to prevent security regressions. Some examples:

  • SSH should enforce MFA authentication
  • Disabled users should be removed from all systems, particularly bastion hosts
  • AWS permissions must prevent services from compromising each other
  • Secrets must be provisioned encrypted
  • ...

2. Increase security maturity

2.1 Help new projects identify threats and controls (RRA, threat models,...)

Risk assessment and threat modeling help people think through failure scenarios they wouldn’t evaluate otherwise. RRAs often leads to architectural changes that are best identified early. Each new project must undergo a 30/60min RRA with one of the member of foxsec to assess the security posture of the project.

2.2 Implement baseline services security standards

Content Security Policy (CSP), HSTS, HPKP, data signature and encryption, input validation, XSS and SQLi protection are part of techniques developers should care about when building new services. foxsec defines services security standards that devs can implement and foxsec tests in TDS.

2.3 Communicate security effectively throughout the organization

Teams need a channel to ask security questions, discuss concerns and share techniques. FoxSec must organize information flow and broadcast to developers, ops and managers. This includes general security best practices, analyzis and actions to take on CVE vulnerabilities, response and communication on incidents.

2.4 Use Mozilla’s bug bounty program

The bug bounty program is a fantastic tool: for a small amount of money, we reward people worldwide for helping us improve our security posture. Most security issues identified in our services come from the bug bounty program. We must ensure that all services are part of the bug bounty program and that triaging is performed regularly. As much as possible, we must assist developers in fixing security issues that are reported through bug bounties.

3. Build core security services

3.1 Sign data that changes the configuration of user agents

We iterate fast, and eventually someone, either us or a partner, is bound to make a mistake and open a door that could put our users at risk. Signing the data we send to our users helps cover that risk. Digital signature for Firefox is a complex topic - not every project can implement it independently - so foxsec must provide the tooling and services to facilitate signing ([autograph](https://github.com/mozilla-services/autograph))

3.2 Monitor our ecosystem for external threats

There are many scenarios in which our users can be at risk because of the fraudulent or careless behavior of a third party. A bad certificate authority could issue a certificate that impersonates us. A careless partner could leak addon signing keys. A web startup could get hacked and leak web push endpoints. We should implement the tools needed to identify fraudulent behavior outside of our organization that impact us, so we can react in a timely manner and protect Firefox users.

3.3 Partner with external firms to monitor our security

We can’t do everything ourselves. External security firms can help us keep an eye on and audit our services. Some of their work may be redundant with current efforts, such as automated security testing, but would help cover the interim. We should evaluate various vendors and partner with the ones that have the best support of our technologies.

Security Checklist

The checklist below is in MARKDOWN format to be copy/pasted into Github issues. 

Risk Management
---------------
* [ ] The service must have performed a Rapid Risk Assessment and have a Risk Record bug (**SVC-RRA**).

Infrastructure rules
--------------------

* [ ] Use [Intermediate TLS](https://wiki.mozilla.org/Security/Server_Side_TLS#Intermediate_compatibility) (**INFRA-TLS**)
* [ ] Set HSTS to 31536000 (1 year) (**INFRA-HSTS**)
* [ ] Set HPKP to 5184000 (60 days) (**INFRA-HPKP**)
  * `Public-Key-Pins: max-age=5184000; pin-sha256="WoiWRyIOVNa9ihaBciRSC7XHjliYS9VwUGOIud4PB18="; pin-sha256="r/mIkG3eEpVdm+u/ko/cwxzOMo1bk4TyHIlByibiA5E="; pin-sha256="YLh1dUR9y6Kja30RrAn7JKnbQG/uEtLMkBgFF2Fuihg="; pin-sha256="sRHdihwgkaib1P1gxX8HFszlD+7/gTfNvuAybgLPNis=";`
    * Start with max-age set to 5 minutes (`max-age=300`) and increase progressively
    * The first two pins are for Digicert EV and DV roots, the last two are for Let's Encrypt X3 and X4 intermediates (LE is only used for backup)
  * [ ] If the service is not hosted under `services.mozilla.com`, it must be manually added to [Firefox's preloaded pins](https://dxr.mozilla.org/mozilla-central/source/security/manager/tools/PreloadedHPKPins.json#184).
* If service has an admin panels, it must:
  * [ ] only be available behind Mozilla VPN (which provides MFA) (**INFRA-ADMINVPN**)
  * [ ] require LDAP authentication (**INFRA-ADMINLDAP**)
  * [ ] enforce a two-man rule on sensitive changes (**INFRA-2MANRULE**)

Coding rules
------------

The following rules apply to all web applications: api and websites.

* [ ] Sign all release tags, and maybe commits (**APP-COMMITSIG**)
  * Developers should [configure git to sign all tags](http://micropipes.com/blog//2016/08/31/signing-your-commits-on-github-with-a-gpg-key/) and upload their PGP fingerprint to https://login.mozilla.com
  * The signature verification will eventually become a requirement to shipping a release to staging & prod: the tag being deployed in the pipeline must have a matching tag in git signed by a project owner. This control is designed to reduce the risk of a 3rd party GitHub integration from compromising our source code.
* [ ] Detailed logging in mozlog format (**APP-MOZLOG**)
  * Business logic must be logged with app specific codes (errno)
  * Access control failures must be logged at WARN level
* [ ] Must have a CSP with (**APP-CSP**)
  * [ ] a report-uri pointing to the service's own `/__cspreport__` endpoint
  * [ ] web APIs should set `default-src` to `none`, disallowing all content rendering
  * [ ] if default-src is not `self`, frame-src should be `none` or only allow specific origins
  * [ ] no use of unsafe-inline or unsafe-eval
* [ ] User data must be escaped for the right context prior to reflecting it (**APP-ESCAPE**)
* [ ] Web APIs must set a non-HTML content-type on all responses, including 300s, 400s and 500s
* [ ] All SQL queries must be parameterized, not concatenated (**APP-SQL**)
* [ ] Apply sensible limits to user inputs, see [input validation](https://wiki.mozilla.org/WebAppSec/Secure_Coding_Guidelines#Input_Validation) (**APP-INPUTVAL**)
* [ ] Enforce Access Controls server-side (**APP-ACL**)
* [ ] Set the Secure flag on [Cookies](https://wiki.mozilla.org/Security/Guidelines/Web_Security#Cookies), and use sensible Expiration and HTTPOnly (**APP-SECCOOKIE**)
* Keep 3rd-party libraries up to date (**APP-DEPS**)
  * [ ]  Use [NSP](https://nodesecurity.io/) or [GreenKeeper](https://greenkeeper.io/ Greenkeeper) for NodeJS applications
  * [ ] Use pip --outdated or [requires.io](https://requires.io/) for Python applications
* [ ] If handling cryptographic keys, must have a mechanism to handle monthly key rotations (**APP-KEYROT**)
  * [ ] All keys must be rotated quarterly.
  * Keys used to sign sessions don't need a rotation mechanism if destroying all sessions is acceptable during.

### Additional websites requirements

The following coding rules only apply to websites, not web apis.

* [ ] Never store passwords, use Firefox Accounts (**APP-IDP**)
* [ ] Forbid Mixed content, always use HTTPS (**APP-MIXCONTENT**)
* [ ] Must have CSRF tokens and manually excluded specific forms (**APP-CSRF**)
* [ ] Should consider having checksums for 3rd-party content via SRI (**APP-SRI**).
  * Trusted 3rd parties, like Google Analytics, don't need SRI. Use your best judgment to decide if a 3rd party script is trustworthy (and assume it is not).
* Set the following security headers (**APP-HEADERS**)
  * [ ] X-Content-Type-Options
  * [ ] X-Frame-Options
  * [ ] X-XSS-Protection
* [ ] Host user uploaded content on a separate domain (e.g. FxA avatar images on firefoxcontent.com, bug attachments on bug<bug ID>.bmoattachments.org)

Data rules
----------

* When storing sensitive user data (like browsing history) on Mozilla servers:
  * [ ] Anonymize it (similar to Tiles) (**DATA-ANON**)
  * [ ] Encrypt it client-side (similar to Sync) (**DATA-CRYPT**)
  * [ ] If user data must be stored non-anonymized and in clear text, you must talk to the security and legal teams about it.
* If the service pushes data to Firefox, like when distributing blacklists or pushing updates, cryptographic signatures must be used. (**DATA-SIGN**)
  * [ ] Addons must use standard AMO signing (**APP-SIGNING**)
  * [ ] Code & Conf must use Content-Signature via [Autograph](https://github.com/mozilla-services/autograph) (**DATA-SIGNING**)

Sites and Services

FoxSec is responsible for the security of the following websites and backend services.

(note: foxsec is not responsible for the security of implementations in firefox, only of the backend services).

ABSearch

Code: absearch

Public Endpoints:

  • search.services.mozilla.com

Addons.mozilla.org

Code:

Public Endpoints:

  • addon.mozilla.org
  • addons.mozilla.org
  • blocklist.addons.mozilla.org
  • builder.addons.mozilla.org
  • controller-review.apk.firefox.com
  • controller.apk.firefox.com
  • services.addons.mozilla.org
  • static.addons.mozilla.net
  • versioncheck-bg.addons.mozilla.org
  • versioncheck.addons.mozilla.org

Product Delivery

Code: go-bouncer

Public Endpoints:

  • download-installer.cdn.mozilla.net
  • download.mozilla.org

AUS/Balrog

Code: balrog

Public Endpoints:

  • aus3.mozilla.org
  • aus4.mozilla.org
  • aus5.mozilla.org
  • aus.mozilla.org

Crash reports (Socorro)

Code: https://github.com/mozilla/socorro/

Public Endpoints:

  • crash-reports-xpsp2.mozilla.com
  • crash-reports.mozilla.com
  • crash-stats.mozilla.com

Firefox Accounts

Code:

Public Endpoints:

  • accounts.firefox.com
  • api.accounts.firefox.com
  • oauth.accounts.firefox.com
  • profile.accounts.firefox.com
  • verifier.accounts.firefox.com

Firefox Sync

Code:

Public Endpoints:

  • *.$region.sync.services.mozilla.com
  • token.services.mozilla.com

Location (MLS)

Code:

Public Endpoints:

  • location.services.mozilla.com
  • location-leaderboard.services.mozilla.com

Marketplace.firefox.com

Code: zamboni

Public Endpoints:

  • marketplace.firefox.com
  • receiptcheck.marketplace.firefox.com
  • static.marketplace.firefox.com

Push

Code:

Public Endpoints:

  • push.services.mozilla.com
  • updates.push.services.mozilla.com

Firefox Settings (Kinto)

Code: https://github.com/Kinto/kinto

Public Endpoints:

  • firefox.settings.services.mozilla.com

Pageshot

Code: https://github.com/mozilla-services/pageshot/

Public Endpoints: pageshot.net

Shield / Normandy

Code:

Public Endpoints: self-repair.mozilla.org

Telemetry

Code:

Public Endpoints:

  • incoming.telemetry.mozilla.org
  • telemetry-experiment.cdn.mozilla.net
  • analysis.telemetry.mozilla.org
  • sql.telemetry.mozilla.org
  • metrics.services.mozilla.com

Test Pilot

Code: testpilot

Public Endpoints:

Tiles/Pingcenter

Code: splice

Public Endpoints:

  • tiles.cdn.mozilla.net
  • tiles.services.mozilla.com

TLS Observatory

Code: tls-observatory

Public Endpoints:

  • tls-observatory.services.mozilla.com

Tracking Protection

Code: shavar

Public Endpoints:

  • shavar.services.mozilla.com
  • tracking.services.mozilla.com
Retrieved from "https://wiki.mozilla.org/index.php?title=Security/FirefoxOperations&oldid=1162627"