QA/Telemetry/TelemetryTestCases

Telemetry Experiments

Test Cases

• Select Telemetry under the Products section on MozTrap

Experiments

Reorder about:newtab tiles

asd

Search Experiment

• Metabug: 1029746 - [meta] Search experiment
• Testing was done in: 1029818 - [Search experiment] QA experiment add-on and sign-off for deployment

Full Query

ID	Summary	Priority	Status
1017806	Telemetry Experiments: builtin support for multiple branches of an experiment	--	VERIFIED
1029189	[UX] Design experience for new default search test	--	RESOLVED
1029776	[Search experiment] Create experiment add-on and define user group (cell) on installation	--	RESOLVED
1029792	[Search experiment] Implement notification UI	--	RESOLVED
1029799	[Search experiment] Define experiment parameters	--	VERIFIED
1029807	[Search experiment] Host update manifest	--	RESOLVED
1029810	[Search experiment] Add the search experiment to the experiments server	--	VERIFIED
1029818	[Search experiment] QA experiment add-on and sign-off for deployment	--	VERIFIED
1030864	[Search experiment] Notify user about the end of the experiment	--	VERIFIED
1031320	[Search experiment] Display notification UI for the correct groups	--	RESOLVED
1033125	Deploy telemetry-experiment-server for search-beta experiment	--	RESOLVED
1036719	[Search experiment] Search experiment should only target en-US	--	VERIFIED

12 Total; 0 Open (0%); 6 Resolved (50%); 6 Verified (50%);

Automatic Translation Experiment

• Metabug: 973271 - Automatic Translation
• Testing was done in: 1022411 - Build Telemetry Experiment for German translation trial

Full Query

ID	Summary	Priority	Status
1021921	Story Breakdown - [Translation] Launch pre-trial of instant translation	--	VERIFIED
1022410	Ensure all translation-related strings in Fx32 Aurora are localized in German	--	VERIFIED
1022411	Build Telemetry Experiment for German translation trial	--	VERIFIED
1022413	Update list of languages supported by automatic translation	--	RESOLVED
1022856	implement translation provider attribution	--	VERIFIED
1032265	Deploy telemetry-experiment-server for translation-aurora experiment	--	VERIFIED

6 Total; 0 Open (0%); 1 Resolved (16.67%); 5 Verified (83.33%);

V2 [In Progress]

Current V2 Status:

Currently in Progress

• Metabug: N/A (No Metabug available, incremental improvements are being made via separate bugs)

Full Query

ID	Summary	Priority	Status
1010397	Experiment manager may undo pending uninstalls	P4	RESOLVED
1012466	exception on shutdown: "Exception calling provider shutdown", NS_ERROR_FAILURE from nsIObserverService.removeObserver from PreviousExperimentProvider.shutdown	--	RESOLVED
1014194	Base implementation: updater addon hotfix	--	VERIFIED
1017806	Telemetry Experiments: builtin support for multiple branches of an experiment	--	VERIFIED
1021002	Nightly test experiment logging doesn't show failures due to incompatible extensions: because the jsfilter isn't working!	--	VERIFIED
1022411	Build Telemetry Experiment for German translation trial	--	VERIFIED
1026850	Experiment system might get active experiment in a confused state if telemetry was disabled	P4	RESOLVED
1026853	Experiment is displayed as "pending removal" in detailed view	--	VERIFIED
1029799	[Search experiment] Define experiment parameters	--	VERIFIED
1029810	[Search experiment] Add the search experiment to the experiments server	--	VERIFIED
1029818	[Search experiment] QA experiment add-on and sign-off for deployment	--	VERIFIED
1030864	[Search experiment] Notify user about the end of the experiment	--	VERIFIED
1032265	Deploy telemetry-experiment-server for translation-aurora experiment	--	VERIFIED

13 Total; 0 Open (0%); 3 Resolved (23.08%); 10 Verified (76.92%);