WebDev/Deployments

This covers deploying of production systems using pip and PyPI.

Goals

• to use pip and Python Package Index (PyPI) whenever possible
• to allow any contributor to easily install and contribute WebDev projects
• to have them deployed quickly and securely internally

Using requirements

Specify packages in the requirements file. We recommend splitting the packages down into separate file for seperate roles, for example: dev.text, prod.txt, test.txt.

All requirements should be pinned, eg:

  foo==0.3

Use of >= or not pinning to a version is not recommended. This can mean broken untested versions of packages being installed.

When running pip use the flag: --no-deps. This ensures that packages will not pull in more untested versions of packages. It also means that the requirements files are a definitive source of packages used. This allows security faster audits of who is using what package.

Note: Packages on PyPI can be removed then replaced so even pinning a version does not mean it can be trusted if you are not using the internal mirror.

Internal package server

When a package is to be used, it will be uploaded to the internal package server.

Existing servers:

• For add-ons and marketplace pyrepo1.

The trusted users who can upload apps to the server are responsible for validating the packages before uploading them. It's up to the uploading user to validate that the package meets security needs.

TODO add in notes about how to upload.

Contributors and external users will continue to use PyPI as normal. Internal production deployments will use this mirror by using the --no-index and --find-links. For example in Marketplace:

   ./venv/bin/pip install --exists-action=w --no-deps --no-index 
   -f http://pyrepo1.addons.phx1.mozilla.com/ -r requirements/prod.txt

Future goals

If we can build RPMs prior to deploying, this will allow security to more easily audit the source of files. Currently services and socorro do this.