Bouncer: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
 
(Remove some meeting information from 2008)
 
(43 intermediate revisions by 18 users not shown)
Line 1: Line 1:
== Abstract ==
Bouncer is a database driven mirror management app that has three main components - a bounce script, a sentry script to check mirrors and files for integrity and a set of web-based administration tools.
Bouncer uses MySQL to organize the Mozilla mirror network.  Its primary responsibility is to know what mirrors are supposed to contain what files and periodically query their availability.  Based on this knowledge, accumulated by a "sentry" script, Bouncer redirects simple user requests to appropriate mirrors based on a random seed that is affected by the mirror's relative bandwidth capabilities.


In short, Bouncer is a smarter way to use an extended mirror network to help users get what they need.
== Project Status  ==


It contains three main pieces:
*[https://svn.osuosl.org/public/bouncer/trunk Public SVN for v1, v2]
* Admin tools to manage the mirror network
*[http://lxr.mozilla.org/mozilla/source/webtools/bouncer Public CVS for v3]
* Sentry; a daemon that queries mirrors for available files and updates their status in the Bouncer database
*[https://github.com/mozilla/tuxedo/ Tuxedo: Django-based admin backend]
* Bouncer; a singular file responsible for taking arguments and redirecting users to the appropriate location


== History ==
== Discussion ==
Bouncer went live at 2pm, November 9th, 2004 and has been serving files since.  It was originally developed as a backup plan, but was pushed to production when the demand for Firefox 1.0 overloaded the existing mirror network.  Utilizing secondary mirrors through Bouncer helped keep downloads flowing in the wake of Mozilla's largest release to date.
* [[MirrorNetwork]] - discussion about the copy of Bouncer currently in use at Mozilla and ideas and brainstorming of how to improve the process, whether it involves Bouncer or not.


Bouncer was originally developed at Oregon State University by Scott Kveton and Mike Morgan.  The intent was to help the Mozilla community meet its growing demands.
# Sentry has many options for tests, testing schedules, trouble notifications and reports.  What should be used? See the Scheduling and Reporting sections on this page: http://wiki.osuosl.org
 
# Sentry runs from one machine.  What happens if that machine loses its connection to the network - suddenly all mirrors are marked bad because they're unreachable?
Additional development on Bouncer has continued into 2005.  K Lars John, a developer at the Oregon State University Open Source Lab, has made great contributions to the project by assisting with database improvements and the migration of Sentry from Perl to Python.  He has helped make Sentry a multi-threaded daemon that will scale much better for an increased mirror network hosting a greater number of projects/files.
 
== Status ==
Bouncer 1.0 is currently directing users to the latest Firefox and Thunderbird releases through the mirror network.  Bouncer 2.0 is nearing the final stages of development and should be deployed before Firefox 1.0.1 is released.
 
== Roadmap ==
=== Bouncer 1.0 ===
* Basic administrative tools
** OS
** Products
** Regions
** Mirrors
** Users
** File Locations
* Sentry
** Basic Perl script to query all locations on all mirrors
** Failed requests would be flagged and would be removed from rotation
* Bouncer Script Parameters
** OS
** Product
** Language
* Known Issues
** Languages; offered as a pass-thru but there was no support for it on the database side of things because it was added post-development
** Versions; 1.0 offers only the latest releases
** Sentry; only queries HTTP mirrors
** Sentry; does not check MD5 sums
 
Bouncer 1.0 was a great start, but lacked some functionality due to time constraints. Since 1.0 we have had improved communications, and developed a solid plan to fix known issues and secure the overall application.
 
1.0 is a temporary solution to aid Mozilla in a time of growth.  It will evolve with time and improve itself through the combined efforts of the community.
 
=== Bouncer 2.0 ===
* Administrative changes
** MD5, SHA1 key storage
** Mirror contact information added
** Mirror policy reviewed and updated
** Versions
** Languages
** Improved database structure (denormalized a bit)
* Sentry
** Converted to Python
** Multi-threaded
** Scalability improved
** Actually downloads file and checks MD5, SHA1
* Bouncer
** Added optional Version parameter; defaults to latest version by date
** Added full language support based on modified DB and admin capabilities
* Development Model
** Integration with Mozilla CVS tree
** Adoption of build process and other Mozilla standards
 
This first joint effort provides better overall security and granularity.  It fixes Language and Version gaps and provides better assurance through improved file checking (MD5,SHA1).
 
Second priorities were improved statistical tools for generating trend reporting.  That was not completed, but will be addressed in 3.0.
 
=== Bouncer 3.0 ===
* Build Features
** Possible automation with build process to streamline releases
** Queue of upcoming releases for build engineers to manage availability of binaries on mirror network
* Improved reporting tools based on download counts and apache logs (stored in separate db)
* UMO
** Possible integration with AUS, PFS, or other update services to leverage mirror network capabilities
** Advanced signing for improved security
 
The future of Bouncer depends on the development of UMO and what role people see it playing in relation to Mozilla's update service.  It will continue to distribute binaries through the mirror network, but how it can better fit into Mozilla's workflow has yet to be determined.  We are working on figuring out how it can help automate and combine builds, updates, etc.  The best is yet to come.

Latest revision as of 16:39, 22 December 2015

Bouncer is a database driven mirror management app that has three main components - a bounce script, a sentry script to check mirrors and files for integrity and a set of web-based administration tools.

Project Status

Discussion

  • MirrorNetwork - discussion about the copy of Bouncer currently in use at Mozilla and ideas and brainstorming of how to improve the process, whether it involves Bouncer or not.
  1. Sentry has many options for tests, testing schedules, trouble notifications and reports. What should be used? See the Scheduling and Reporting sections on this page: http://wiki.osuosl.org
  2. Sentry runs from one machine. What happens if that machine loses its connection to the network - suddenly all mirrors are marked bad because they're unreachable?
Retrieved from "https://wiki.mozilla.org/index.php?title=Bouncer&oldid=1110358"