BMO/ElasticSearch

From MozillaWiki
< BMO
Jump to: navigation, search

A Public ElasticSearch Cluster with Bugzilla Data

Objective

Provide the public with an easy-to-query historical view of Bugzilla bugs.

Justification

The BMO/ES cluster is important for managing many of Mozilla's projects: It is the underlying active data system that is used by a multitude of dashboards to track various project metrics. It is capable of providing both high level aggregates over the million+ issues, providing detailed historical views into those issues, and it provides that information fast.

Let's Use It!

BMO/ES is a service, which means you can use it right now! You probably already have your browser open, so lets send it queries:

Examples

Minimum required to hit ES cluster on Github
Minimum required to pull comments on Github
Simple Bug Count on Github
Can you reach the private cluster? on Github

Basic walk-through

https://github.com/klahnakoski/Qb/blob/master/docs/BZ_Tutorial.md

Code

Backend ETL Code: https://github.com/klahnakoski/Bugzilla-ETL
Frontend Examples: https://github.com/klahnakoski/MoDevMetrics/tree/dev

Hacking/Contributors

This project needs your help! If you love data munging and dealing with complexity; and you get satisfaction that people will build atop your hard work every day, then this project is for you!

Contacts

  • IRC: ekyle@irc.mozilla.org (in #ateam and #bteam)
  • email: klahnakoski@mozilla.com

Good First Bugs

ID Summary Status Resolution Assigned to

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);


Big bugs

ID Summary Status Resolution Assigned to
922905 Add code to trigger full ETL ASSIGNED Kyle Lahnakoski [:ekyle] (klahnakoski)
945854 https://bugzilla.mozilla.org/show_bug.cgi?id=749568 does not clear review flag properly NEW
945862 Add BZ Patch Metadata to ES NEW
961588 Alias Analysis to put less preasure on BZ Database NEW
962882 Getting *real* bug history is hard NEW
986508 Alias analysis should use ES as datastore, not file NEW
987706 Inconsistent value fields not handled well enough NEW
997228 Renaming flags will destroy bug history for those flag values NEW
1028950 Find metric to detect imminent failure of ES cluster NEW
1067395 Use 'X' instead of 'D' to mark cleared flags NEW
1110912 New Index with bug current state only NEW
1125904 Blocked/blocked_by/depends_on is too confusing NEW
1160411 Send alert when ETL stops NEW

13 Total; 13 Open (100%); 0 Resolved (0%); 0 Verified (0%);


Other Reading