Metrics/Telemetry 2013-02-14

Status

Harsha worked on this during the week and finished most of it today.
Ability to filter on simple measures or histograms, will add ability to filter on input variables like channel and version
Will let user specify list of attributes that they want to return
Pre-release version is available, some further tweaks to be made Friday and Monday.

Depends on BIV-114
This is a test case for the simple queries feature, will be delivered early next week.

Development wrapping up. Finished cdf, still cleanup and testing that isn't finished.
Want to see where we want to use new chart - currently in histograms dashboard, may want it in evolution as well
Will prompt perf team for review once on staging
Pushing to staging on 15th for verification and can deploy shortly after.

Need to discuss options for implementation with Xavier
Looking to deliver capability through the simple query script rather than dashboard

Taras agrees with prototyping things with scripts before going to dashboards

METRICS-1366 Submission spike

Are there any more requests for Nathan or Irving?
Daniel is working on generating statistics
- https://docs.google.com/spreadsheet/ccc?key=0AtdL1GrYQUbldGVadzlrcnJSV0R4VGJKSkZZQXQxSEE#gid=0
Harsha is going to do another run for Nathan's request
Requests from Firefox versions
- Lots of FF10-12 sending requests
- A few hundred for FF13-15
- Starting with FF16 we're back in the 1000s
- This is not strictly a problem with old builds
- FF18 seeing 40,000 pings per day where id has already been seen that day
Note: this is not the same problem that we saw with the original submission spike
- Submission spike is an increase in the number of docs that are canonically stored in hbase, only store one copy of an id for a given day in hbase
- Thought is that Q3 had saved submission fix that went live in Jan in FF18, started seeing spikes in saved submissions all submitted at once after update
- Trend looks like it's settling down, another week of data to call that fully done
Main concern at this point is the multiple submissions per client
Implemented a write limit?
- Should have a way to ask client to throttle submissions
- Suggestion: Whenever have a non 200 response don't send any more sessions that day, can also look at purging sessions
- Daniel to file bug

Xavier was doing a lot of Telemetry work, is there a plan to fill the gap?
- Looking to replace the head count
- Harsha has been coming up to speed on Xavier's work
- Mark has also been working with Xavier, more on server side, will look at pig once he is comfortable on the server side
- Want to have at least two resources who can help with scripting