Confirmed users
539
edits
CloudServices/DataPipeline: Difference between revisions
m
formatting
m (tweak wording) |
m (formatting) |
||
| Line 31: | Line 31: | ||
= Work Queue = | = Work Queue = | ||
To Do/Doing categories need to be logged in bugzilla. | To Do/Doing categories need to be logged in bugzilla. | ||
== Risks and Open Questions == | === Risks and Open Questions === | ||
* Send something to dev-planning? [kparlante, telliot] | * Send something to dev-planning? [kparlante, telliot] | ||
* Old-FHR data through pipeline? Yes/No: [telliot] | * Old-FHR data through pipeline? Yes/No: [telliot] | ||
* Deletes & legal policy [telliot, mreid to provide cost estimate] | * Deletes & legal policy [telliot, mreid to provide cost estimate] | ||
== Needs more discussion == | === Needs more discussion === | ||
* Maintaining a sample data set for faster queries | * Maintaining a sample data set for faster queries | ||
* Implement a specific flag to determine if data gets warehoused or not | * Implement a specific flag to determine if data gets warehoused or not | ||
| Line 49: | Line 49: | ||
** example: does fxa/sync increase browser usage? | ** example: does fxa/sync increase browser usage? | ||
== To Do == | === To Do === | ||
* Q4 telemetry: (re) implement telemetry monitoring dashboards [?] | * Q4 telemetry: (re) implement telemetry monitoring dashboards [?] | ||
* Q1 BI: define schema for data warehouse (talk to jjensen) [kparlante] | * Q1 BI: define schema for data warehouse (talk to jjensen) [kparlante] | ||
| Line 62: | Line 62: | ||
** follow up with saptarshi for more complex algorithm | ** follow up with saptarshi for more complex algorithm | ||
== Doing == | === Doing === | ||
* Opsify stack [whd] | * Opsify stack [whd] | ||
* Q4 telemetry: Send telemetry data through the pipeline [mreid] | * Q4 telemetry: Send telemetry data through the pipeline [mreid] | ||
| Line 68: | Line 68: | ||
* risk mitigation: Estimate cost of “full scan” DWH query [mreid] | * risk mitigation: Estimate cost of “full scan” DWH query [mreid] | ||
* risk mitigation: Estimate cost of single DWH delete [mreid] | * risk mitigation: Estimate cost of single DWH delete [mreid] | ||
== Done == | |||
=== Done === | |||
* Parallelize sandbox filters (eg FHRSearch) [trink] | * Parallelize sandbox filters (eg FHRSearch) [trink] | ||
* Enable Lua JIT [trink] | * Enable Lua JIT [trink] | ||