Confirmed users
487
edits
QA/Mobile/TelemetryTestPlan: Difference between revisions
m
type
Rpappalardo (talk | contribs) (clarify submit data) |
Rpappalardo (talk | contribs) m (type) |
||
| Line 50: | Line 50: | ||
=== Description === | === Description === | ||
For each ping scenario, we will conduct a single end-to-end test. Each test can be broken down into three high-level steps: | For each ping scenario, we will conduct a single end-to-end test. Each test can be broken down into three high-level steps: | ||
# [[#Data Collection|Data Collection]] | # [[#Data Collection & Submission|Data Collection & Submission]] | ||
#* Ping data is collected on the client (and stored locally on disk). Some data is recorded automatically (i.e. [https://github.com/mozilla-mobile/android-components/blob/master/components/service/glean/docs/pings/baseline.md baseline] and [https://github.com/mozilla-mobile/android-components/blob/master/components/service/glean/docs/pings/metrics.md metrics] pings). Other data is collected when certain specific actions are performed by the user. The latter are referred to as "events" (i.e. [https://github.com/mozilla-mobile/android-components/blob/master/components/service/glean/docs/pings/events.md events] pings). | #* Ping data is collected on the client (and stored locally on disk). Some data is recorded automatically (i.e. [https://github.com/mozilla-mobile/android-components/blob/master/components/service/glean/docs/pings/baseline.md baseline] and [https://github.com/mozilla-mobile/android-components/blob/master/components/service/glean/docs/pings/metrics.md metrics] pings). Other data is collected when certain specific actions are performed by the user. The latter are referred to as "events" (i.e. [https://github.com/mozilla-mobile/android-components/blob/master/components/service/glean/docs/pings/events.md events] pings). | ||
#* Our test coverage must ensure we verify that [[#Data Collection|data collection | #* Our test coverage must ensure we verify that [[#Data Collection & Submission|data collection and submission]] has taken place locally for each of the ping scenarios. We can monitor this local activity with the logcat tool (built into Android Studio). | ||
#* Using the [https://github.com/mozilla-mobile/android-components/tree/master/components/service/glean#debugging-products-using-glean Glean CLI tool], we'll tag and force data to be submitted to the telemetry server immediately, rather than waiting for a scheduled trigger event(s) or for thresholds to be met, etc. | #* Using the [https://github.com/mozilla-mobile/android-components/tree/master/components/service/glean#debugging-products-using-glean Glean CLI tool], we'll tag and force data to be submitted to the telemetry server immediately, rather than waiting for a scheduled trigger event(s) or for thresholds to be met, etc. | ||
# [[#Data Integrity|Data Integrity Verification]] | # [[#Data Integrity|Data Integrity Verification]] | ||
| Line 71: | Line 70: | ||
# [[#Test Setup|Test Setup]] | # [[#Test Setup|Test Setup]] | ||
# [[#Data Collection|Data Collection | # [[#Data Collection & Submission|Data Collection & Submission]] | ||
# [[#Data Integrity|Data Integrity Verification]] | # [[#Data Integrity|Data Integrity Verification]] | ||
| Line 188: | Line 186: | ||
* [[#Verify_Common_Info|common fields info]] | * [[#Verify_Common_Info|common fields info]] | ||
Verify events that were performed in the [[#Data Collection|Data Collection]] section have arrived in Dashboard | Verify events that were performed in the [[#Data Collection & Submission|Data Collection & Submission]] section have arrived in Dashboard | ||
NOTE: | NOTE: | ||