BMO team (dkl, glob, mcote), ebryn (contract developer on front end), peterbe (advisor from web engineering)
Goals & Considerations
Provide a push-based notification system to inform clients of changes to bugs. Plan for scalability by minimizing server load and time from change to notification.
Providing details of what has changed is not necessary in the notification itself, although preferably this information would be available somehow, perhaps in a separate call.
Re-implementing Bugzilla's permissions system is not an option. It is complex enough that changing the current model would be major surgery (and would further diverge BMO from upstream), and maintaining two parallel implementations would incur maintenance costs and be error prone.
Notifications should not include changes to dependent/related bugs. This is harder to track based on the current Bugzilla database schema, and a properly designed system should be able to track them indepedently at the client's discretion. The exception here would be adding or removing dependencies, e.g. a notification will be sent for both bug X and Y if X is set to depend on Y.
Design and Approach
We will implement a separate server that polls the database directly for changes to bugs and passes on *only the ID and change time* of the changed bug to its clients. The clients can then use the main Bugzilla REST API to determine the exact changes, which will enforce permissions as usual.
Clients will be able to subscribe to one or more bugs, and will be able to get a list of their subscriptions and unsubscribe from them. Subscriptions will only be valid for the current session; if a client disconnects, they will have to resubscribe to their bugs.
Important note: this whole design relies upon the idea that knowing the ID of a changed bug is not a security risk. This should be reasonable, given that the only information that is conveyed is that some bug has changed. One could use this information to determine the frequency of changes to a particular bug over some time frame, and hence perhaps an increased interest in a particular bug, but the changes could be to anything--main bug fields, comments, tracking flags, dependencies, etc.
Conceptually, there are three parts:
- Database poller. There would be exactly one process that frequently polls the database for changes (period TBD but on the order of seconds, not minutes). This would keep the time of the last poll in memory and would ask for only the ID of all bugs changed since the time of the last poll.
- We will use a separate table to hold the notifications, which will be written by a Bugzilla extension called PushNotify. The poller will read entries from this table and publish messages on pulse.
- TCP servers. There would be one or more processes acting as servers that accept client connections and maintain them indefinitely. WebSockets is the preferred protocol for easy integration with browsers. These servers would listen for notifications from the database poller and fan out notification messages to all clients according to subscriptions. For scalability, multiple server processes could be launched with a load-balancer (such as Zeus) spreading out connections amongst them. Note that the server will receive notifications for *all* bugs, since we don't expect there to be many servers. It will be up to the server to implement a subscription protocol and keep track of them.
- Messaging middleware. We are using Pulse for a couple reasons: it's already set up and easily extended, and we can eventually support a variety of transports/servers.
System diagram (courtesy of contributor musingmario):
Work on the prototype is being tracked in bug 923849.
The changes are written by Bugzilla using the ZPushNotify extension (http://bzr.mozilla.org/bmo/4.2/files/head:/extensions/ZPushNotify/).
The changes are picked up and delivered to Pulse by the Bugzilla Simple Shim (https://hg.mozilla.org/automation/pulseshims/ - relevant files are bugzilla_simple_shim.py, requirements_bugzilla_simple_shim.txt, and the bottom of config.ini.example).
The Pulse messages are consumed by bugzfeed (https://github.com/mozilla/bugzfeed).
If you want to hack on the system, see the HACKING.md file in bugzfeed (https://github.com/mozilla/bugzfeed/blob/master/HACKING.md).