ReleaseEngineering/Applications/Talos Dirty Profiles
Several of the talos tests run against dirty Firefox profiles of various sizes, to test related timings. Those profiles are downloaded dynamically, and need to be kept up to date so that history records are not expired by the browser.
- Sqlite (as of bug 600980, 3.7.0 or later)
- Python with sqlite bindings
- Django (used within the generation scripts - dep could be removed if necessary)
- Web server
- System for batch processing (cruncher) meeting requirements above
- dm-wwwbuild01 (http://build.mozilla.org)
- Requires automated rsync access from cruncher
This application has no security concerns - the contents of the profiles are not made public, but do not contain any sensitive data.
No monitoring at this time.
dm-wwwbuild01 serves up the talos profiles from http://build.mozilla.org/talos/profiles. Access to this directory is limited to the build network via Apache configuration.
Profiles are generated on cruncher, from a crontask that runs at midnight every night, from the talos user's private crontab:
0 0 * * * (/usr/local/bin/python /home/talos/public_html/updateGenerated.py) >> /home/talos/public_html/reporter.log && mv /home/talos/public_html/dirtyDBs.zip /var/www/html/talos_profiles/. && mv /home/talos/public_html/dirtyMaxDBs.zip /var/www/html/talos_profiles/. && cp /var/www/html/talos_profiles/dirtyMaxDBs.zip /home/talos/public_html/. && cp /var/www/html/talos_profiles/dirtyDBs.zip /home/talos/public_html/
The updateGenerated.py script is in http://hg.mozilla.org/build/tools/buildfarm/utils/
The original profiles were generated, I believe, by http://hg.mozilla.org/build/tools/buildfarm/utils/generator/, but are probably best considered precious cargo at this point.
Profiles are copied from cruncher to dm-wwwbuild01 every 2 hours, at 10 minutes past the hour, from root's private crontab (crontab -l).
This project is no longer developed. Note that any changes to the profiles may cause timing changes and must be coordinated carefully with the platform folks.