Confirmed users
513
edits
Auto-tools/Projects/PublicES: Difference between revisions
Jump to navigation
Jump to search
Auto-tools/Projects/PublicES (view source)
Revision as of 03:11, 20 November 2013
, 20 November 2013no edit summary
Klahnakoski (talk | contribs) |
Klahnakoski (talk | contribs) No edit summary |
||
| Line 63: | Line 63: | ||
In the end we have a high speed ETL solution that is easy to install and execute. There are plenty of improvements that can be made, and definitely in the area of more threads and more multiple processes. But those can wait while we deploy. | In the end we have a high speed ETL solution that is easy to install and execute. There are plenty of improvements that can be made, and definitely in the area of more threads and more multiple processes. But those can wait while we deploy. | ||
= SecReview (20 November 2013, incomplete) = | |||
== Introduction == | |||
Public Bugzilla data in a publicly accessible ElasticSearch cluster! | |||
* '''Overview''' https://wiki.mozilla.org/Auto-tools/Projects/PublicES | |||
* '''Architecture''' https://bugzilla.mozilla.org/attachment.cgi?id=828667 | |||
* '''History''' https://wiki.mozilla.org/index.php?title=Bugzilla_Anthropology/2013-01-29 | |||
=== Goal of Feature, what is trying to be achieved (problem solved, use cases, etc) === | |||
* Provide '''public''' fast cache of BZ data to: | |||
*# demonstrate current work | |||
*# allow community to build tools | |||
*#* https://github.com/okononen/dash | |||
*#* http://www.joshmatthews.net/bugsahoy/ | |||
*#* http://harthur.github.io/bzhome/ | |||
*#* http://pike.github.io/beta-dash/ | |||
*# allow community to analyze trends, patterns | |||
=== What solutions/approaches were considered other than the proposed solution? === | |||
* Tried to publicize the existing ES cluster information (private bugs with no comments of short description), but there was concern the CC list may reveal the bug's security category (https://bugzilla.mozilla.org/show_bug.cgi?id=823303) | |||
* Using the BZ-API directly requires sophisticated caching, which appears to stall all attempts at making snappy dashboards. | |||
=== Why was this solution chosen? === | |||
* ElasticSearch is very fast | |||
* Direct DB access leverages existing code | |||
* Direct DB access put no load on Bugzilla app | |||
* Proven to work with business intelligence queries, which demand fast aggregate data over thousands of bugs https://wiki.mozilla.org/Bugzilla_Anthropology/2013-01-29) | |||
=== Any security threats already considered in the design and why? === | |||
* Private bug data leaking into public cluster | |||
* ElasticSearch was not meant for direct public access, proxy added (https://bugzilla.mozilla.org/show_bug.cgi?id=879833) | |||