Firefox/Python 3 Migration
- 1 Why This is Important
- 2 How We Get There
- 3 Getting Involved
- 4 How to Port the Code
- 5 References
Why This is Important
In mozilla-central there are over 3500 Python files (excluding third party files), comprising roughly 230k lines of code. Most of it is Python 2.
Python 2 will reach it's End Of Life on January 1st 2020. This page collects links and resources to assist the migration of the Firefox ecosystem off of Python 2 and onto Python 3.
How We Get There
As of today (May 2019) we require both the Python 2 and Python 3 interpreters to be installed to build and develop Firefox.
- Make Python 3 porting and development safe by adding a Python 3 option to the developer toolchain. (mach lint, coverage.py, parts of the test suite)
- Make key support libraries run in both Python 2 and Python 3. (mach-core)
- Build interpreter switching mechanisms into the toolchain so we can run sub-components in just 2 or just 3. (mach sub-commands, py_action)
- Port Python 2-only components to Python 3 piecemeal.
- Remove Python 2 compatibility. (When that might happen needs discussion. Possibly sometime in 2020.)
Find the others
Most discussion happens in the #python channel on Slack (NDA'd Mozillians only).
What you'll need
You will need a working Firefox build environment.
Where to start:
Help port mach-core to Python 3 (bug 1473498)
- Edit python/mach/mach/test/python.ini to remove the skip-if
- Run: ./mach python-test --python 3 python/mach
- Fix the errors until all tests pass
Remove more of the "excluded" paths from the py2 and py3 linters
Remove a line from https://searchfox.org/mozilla-central/source/tools/lint/py3.yml or https://searchfox.org/mozilla-central/source/tools/lint/py2.yml. Run the linter, see if the module passes.
Run all python source tests with Python 3 and see what breaks
Make sure each of the tasks listed in https://searchfox.org/mozilla-central/source/taskcluster/ci/source-test/python.yml has Python 3 specified, and do a try push, enabling all the python source tests.
See what breaks, then fix it!
See the #References for links to other ideas and tracking bugs.
How to Port the Code
If you haven't ported modules before (or need a refresher), read the Porting Guide on python.org. It has a good outline of the porting process.
Porting modules in the Firefox source tree
- We've already standardized on Python 2.7 across the tree.
- Make sure you can test the code you're working on with python3!
- If using `tox`, then adding `py35`, and `py36` to the list of environments should be sufficient.
- If using |mach python-test|, make sure the relevant manifest doesn't skip-if Python 3 and run |mach python-test --python 3 <path>|.
- `futurize` Stage 1 fixes can be used to transform code into code that's compatible with both python2.7 and python3.
- The `six` module is available in-tree, feel free to use it as a dependency.
- Black can format Python 2 code and Python 3 code (but it requires Py3.6+ to run).
Cheat Sheet: Writing Python 2-3 compatible code. Note: misses some advanced changes, like porting __eq__() to __hash__() and changes to the slice interface.
Bug 1496527 tracks the migration