Auto-tools/Projects/OrangeFactor/ElasticSearch
The data for bugs and logs is stored in ElasticSearch.
Contents
Documentation
The ElasticSearch query language is described at [1].
Bug Data
Data for a single occurrence of a bug looks like this:
_source: { buildname: "Rev3 MacOSX Leopard 10.5.8 mozilla-central opt test mochitest-other" machinename: "talos-r3-leopard-038" os: "osx" date: "2010-12-14" type: "Mochitest" debug: "false" starttime: "1292378723" logfile: "1292378723.1292379802.29699.gz" tree: "mozilla-central" rev: "abe884259481" who: "philringnalda@gmail.com" bug: "614643" }
Bug Queries
Some sample bug queries are shown below. Most ES queries are conducted by sending a JSON document via an HTTP GET to the db; the JSON document describes the search terms. You can execute these queries from the command-line using curl, which is the format used in these examples.
In ES, all queries will return a default of 10 matches at most. You can alter this behavior by changing the "from" and "size" fields in the queries below.
Get all bugs in date range for mozilla-central
curl -XGET 'http://cm-metricsetl03.mozilla.org:9200/bugs/bug_info/_search?pretty=true' -d ' { "from": 0, "size": 20, "query": { "filtered": { "query": { "field": { "tree": "mozilla-central" }}, "filter": { "range": { "date": { "from": "2010-12-21", "to": "2010-12-22" } } } } } } '
Get all occurrences of a specific bug in date range for m-c
curl -XGET 'http://cm-metricsetl03.mozilla.org:9200/bugs/bug_info/_search?pretty=true' -d ' { "from": 0, "size": 5, "query": { "filtered": { "query": { "bool": { "must": [ { "field": { "bug": "614643" } }, { "field": { "tree": "mozilla-central" } } ] } }, "filter": { "and": { "filters": [ { "range": { "date": { "from": "2010-12-21", "to": "2010-12-22" } } } ] } } } } } '
Getting a count for a query
To get a count of items that will be returned by a query, you issue the query to a url that has _count
in place of _search
, and omits the "query" element which wraps the query (the "query" is implied for _count
).
For example, the query to retrieve the first 10 records for all the bugs in mozilla-central looks like this:
curl -XGET 'http://cm-metricsetl03.mozilla.org:9200/bugs/bug_info/_search?pretty=true' -d ' { "from": 0, "size": 10, "query": { "field": { "tree": "mozilla-central" } } } '
To count the number of records that exist for this query, you would use:
curl -XGET 'http://cm-metricsetl03.mozilla.org:9200/bugs/bug_info/_count?pretty=true' -d ' { "field": { "tree": "mozilla-central" } } '
This might return:
{ "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 773, "max_score" : 1.7568314, "hits" : [ ] } }
The value of json["hits"]["total"] is the total number of items that exist for that query.