ReleaseEngineering/Applications/Talos Dirty Profiles

From MozillaWiki
Jump to: navigation, search

Application Description

Several of the talos tests run against dirty Firefox profiles of various sizes, to test related timings. Those profiles are downloaded dynamically, and need to be kept up to date so that history records are not expired by the browser.

Requirements

  • Rsync
  • Sqlite (as of bug 600980, 3.7.0 or later)
  • Python with sqlite bindings
  • Django (used within the generation scripts - dep could be removed if necessary)
  • Web server

Resources

  • System for batch processing (cruncher) meeting requirements above
  • dm-wwwbuild01 (http://build.mozilla.org)
    • Requires automated rsync access from cruncher

Security

This application has no security concerns - the contents of the profiles are not made public, but do not contain any sensitive data.

Monitoring

No monitoring at this time.

Deployment

Web Server

dm-wwwbuild01 serves up the talos profiles from http://build.mozilla.org/talos/profiles. Access to this directory is limited to the build network via Apache configuration.

Profile Generation

Profiles are generated on cruncher, from a crontask that runs at midnight every night, from the talos user's private crontab: 

0 0 * * * (/usr/local/bin/python /home/talos/public_html/updateGenerated.py) >> /home/talos/public_html/reporter.log && mv /home/talos/public_html/dirtyDBs.zip /var/www/html/talos_profiles/. &&  mv /home/talos/public_html/dirtyMaxDBs.zip /var/www/html/talos_profiles/. && cp /var/www/html/talos_profiles/dirtyMaxDBs.zip /home/talos/public_html/. && cp /var/www/html/talos_profiles/dirtyDBs.zip /home/talos/public_html/

The updateGenerated.py script is in http://hg.mozilla.org/build/tools/buildfarm/utils/

The original profiles were generated, I believe, by http://hg.mozilla.org/build/tools/buildfarm/utils/generator/, but are probably best considered precious cargo at this point.

Profile Copies

Profiles are copied from cruncher to dm-wwwbuild01 every 2 hours, at 10 minutes past the hour, from root's private crontab (crontab -l).

Development

This project is no longer developed. Note that any changes to the profiles may cause timing changes and must be coordinated carefully with the platform folks.

Retrieved from "https://wiki.mozilla.org/index.php?title=ReleaseEngineering/Applications/Talos_Dirty_Profiles&oldid=352365"