B2G/QA/2.2/2015-04-27Status - Revision history

Joshwalker: /* Functional Testing */

2015-04-27T14:56:09Z

Functional Testing

← Older revision		Revision as of 14:56, 27 April 2015
Line 100:		Line 100:
	* RTL (Marigold)		* RTL (Marigold)
	** Health Status: Green		** Health Status: Green
	** Details: 3 blockers http://mzl.la/1Juz6oH		** Details: 2 blockers http://mzl.la/1Juz6oH

	* Cut, Copy, and Paste (Gerry, Edward, Teri)		* Cut, Copy, and Paste (Gerry, Edward, Teri)

Echang: /* V2.2 Highlight & Risk */

2015-04-27T13:35:24Z

V2.2 Highlight & Risk

← Older revision		Revision as of 13:35, 27 April 2015
Line 1:		Line 1:
	== V2.2 Highlight & Risk ==		== V2.2 Highlight & Risk ==
	* Highlight		* Highlight
	** V2.2 FC moved to 2015/04/29		** V2.2 FC moved to 2015/04/29.
	** NEXUS-5-L 5.1 PVT build MTBF 34.42 hrs		** NEXUS-5-L 5.1 PVT build MTBF 38.5 hrs & counting.
	** NEXUS-5-L 5.1 Regression test ~~started from 2015/04/14 for 2 wks~~		** FLAME-KK 4.4 PVT build MTBF 34.42 hrs.
			** NEXUS-5-L 5.1 Regression test done.
	** 48 bugs with blocking=2.2+ http://mzl.la/1CDEaqi		** 48 bugs with blocking=2.2+ http://mzl.la/1CDEaqi
	** 22 bugs with blocking=2.2? http://mzl.la/1CDEKV4		** 12 bugs with blocking=2.2? http://mzl.la/1CDEKV4

	== [[Release_Management/FirefoxOS/2_2_Schedule\|V2.2 Milestone]] ==		== [[Release_Management/FirefoxOS/2_2_Schedule\|V2.2 Milestone]] ==

Echang: /* 2nd Run (2015/04/14) */

2015-04-27T10:33:58Z

2nd Run (2015/04/14)

← Older revision		Revision as of 10:33, 27 April 2015
Line 51:		Line 51:
	==== Regression Testing ====		==== Regression Testing ====
	== 2nd Run (2015/04/14) ==		== 2nd Run (2015/04/14) ==
	* '''NEXUS-5-L 2nd Run (focus on essential functionality)'''		* '''NEXUS-5-L 2nd Run (focus on essential functionality)''' <span style="color:#ffffff; background:#008000"> Done </span>
	** Test cases http://mzl.la/1D76gXG		** Test cases http://mzl.la/1D76gXG

Line 62:		Line 62:
	** http://mzl.la/1cYZZau (Bugs related to software home button, privacy or ui polish will be skipped for now)		** http://mzl.la/1cYZZau (Bugs related to software home button, privacy or ui polish will be skipped for now)

	* '''FLAME-KK 2nd Run (focus on dsds and gaia apps)'''		* '''FLAME-KK 2nd Run (focus on dsds and gaia apps)''' <span style="color:#ffffff; background:#008000"> Done </span>
	** Test cases http://mzl.la/1aKYIC8		** Test cases http://mzl.la/1aKYIC8

Echang: /* 1st Run (2015/03/03) */

2015-04-27T10:33:06Z

1st Run (2015/03/03)

← Older revision		Revision as of 10:33, 27 April 2015
Line 74:		Line 74:

	== 1st Run (2015/03/03) ==		== 1st Run (2015/03/03) ==
	* RTL/V2.2 Flame-KK <span style="color:#~~000000~~; background:#008000"> Done </span>		* RTL/V2.2 Flame-KK <span style="color:#ffffff; background:#008000"> Done </span>
	** [https://moztrap.mozilla.org/results/runs/?&pagenumber=1&pagesize=20&sortfield=start&sortdirection=desc&filter-status=active&filter-name=rtl+full+test Moztrap result] - [https://moztrap.mozilla.org/results/cases/?&pagenumber=1&pagesize=100&sortfield=created_on&sortdirection=desc&filter-resultstatus=failed&filter-run=6626 76 fail], 326 pass, check with Josh for detail.		** [https://moztrap.mozilla.org/results/runs/?&pagenumber=1&pagesize=20&sortfield=start&sortdirection=desc&filter-status=active&filter-name=rtl+full+test Moztrap result] - [https://moztrap.mozilla.org/results/cases/?&pagenumber=1&pagesize=100&sortfield=created_on&sortdirection=desc&filter-resultstatus=failed&filter-run=6626 76 fail], 326 pass, check with Josh for detail.

	* Full Regression/V2.2 Flame-KK <span style="color:#~~000000~~; background:#008000"> Done </span>		* Full Regression/V2.2 Flame-KK <span style="color:#ffffff; background:#008000"> Done </span>
	** [http://mzl.la/1wtR60Y Moztrap result] - [https://moztrap.mozilla.org/results/cases/?&pagenumber=1&pagesize=100&sortfield=created_on&sortdirection=desc&filter-resultstatus=failed&filter-run=6551 55 fail], 3426 pass, 100% complete		** [http://mzl.la/1wtR60Y Moztrap result] - [https://moztrap.mozilla.org/results/cases/?&pagenumber=1&pagesize=100&sortfield=created_on&sortdirection=desc&filter-resultstatus=failed&filter-run=6551 55 fail], 3426 pass, 100% complete

	* Partial Regression/V2.2 Nexus-5-L <span style="color:#~~000000~~; background:#008000"> Done </span>		* Partial Regression/V2.2 Nexus-5-L <span style="color:#ffffff; background:#008000"> Done </span>
	** [http://mzl.la/1EXfMQQ Moztrap result] - [https://moztrap.mozilla.org/results/cases/?&pagenumber=1&pagesize=100&sortfield=created_on&sortdirection=desc&filter-resultstatus=failed&filter-run=6552 30 fail], 1216 pass, 100% complete		** [http://mzl.la/1EXfMQQ Moztrap result] - [https://moztrap.mozilla.org/results/cases/?&pagenumber=1&pagesize=100&sortfield=created_on&sortdirection=desc&filter-resultstatus=failed&filter-run=6552 30 fail], 1216 pass, 100% complete
	** [http://mzl.la/18T1AMy Bug list]		** [http://mzl.la/18T1AMy Bug list]

	* Bluetooth PTS/V2.2 Flame-KK <span style="color:#~~000000~~; background:#008000"> Done </span>		* Bluetooth PTS/V2.2 Flame-KK <span style="color:#ffffff; background:#008000"> Done </span>
	** [https://bugzilla.mozilla.org/showdependencytree.cgi?id=1139349&hide_resolved=1 Bug Meta dependency tree] - 0 open bugs		** [https://bugzilla.mozilla.org/showdependencytree.cgi?id=1139349&hide_resolved=1 Bug Meta dependency tree] - 0 open bugs

Echang: /* 1st Run (2015/03/03) */

2015-04-27T10:29:27Z

1st Run (2015/03/03)

← Older revision		Revision as of 10:29, 27 April 2015
Line 74:		Line 74:

	== 1st Run (2015/03/03) ==		== 1st Run (2015/03/03) ==
	* RTL/V2.2 Flame-KK <span style="color:#000000; background:#~~FFFF00~~"> Done </span>		* RTL/V2.2 Flame-KK <span style="color:#000000; background:#008000"> Done </span>
	** [https://moztrap.mozilla.org/results/runs/?&pagenumber=1&pagesize=20&sortfield=start&sortdirection=desc&filter-status=active&filter-name=rtl+full+test Moztrap result] - [https://moztrap.mozilla.org/results/cases/?&pagenumber=1&pagesize=100&sortfield=created_on&sortdirection=desc&filter-resultstatus=failed&filter-run=6626 76 fail], 326 pass, check with Josh for detail.		** [https://moztrap.mozilla.org/results/runs/?&pagenumber=1&pagesize=20&sortfield=start&sortdirection=desc&filter-status=active&filter-name=rtl+full+test Moztrap result] - [https://moztrap.mozilla.org/results/cases/?&pagenumber=1&pagesize=100&sortfield=created_on&sortdirection=desc&filter-resultstatus=failed&filter-run=6626 76 fail], 326 pass, check with Josh for detail.

	* Full Regression/V2.2 Flame-KK <span style="color:#000000; background:#~~FFFF00~~"> Done </span>		* Full Regression/V2.2 Flame-KK <span style="color:#000000; background:#008000"> Done </span>
	** [http://mzl.la/1wtR60Y Moztrap result] - [https://moztrap.mozilla.org/results/cases/?&pagenumber=1&pagesize=100&sortfield=created_on&sortdirection=desc&filter-resultstatus=failed&filter-run=6551 55 fail], 3426 pass, 100% complete		** [http://mzl.la/1wtR60Y Moztrap result] - [https://moztrap.mozilla.org/results/cases/?&pagenumber=1&pagesize=100&sortfield=created_on&sortdirection=desc&filter-resultstatus=failed&filter-run=6551 55 fail], 3426 pass, 100% complete

	* Partial Regression/V2.2 Nexus-5-L <span style="color:#000000; background:#~~FFFF00~~"> Done </span>		* Partial Regression/V2.2 Nexus-5-L <span style="color:#000000; background:#008000"> Done </span>
	** [http://mzl.la/1EXfMQQ Moztrap result] - [https://moztrap.mozilla.org/results/cases/?&pagenumber=1&pagesize=100&sortfield=created_on&sortdirection=desc&filter-resultstatus=failed&filter-run=6552 30 fail], 1216 pass, 100% complete		** [http://mzl.la/1EXfMQQ Moztrap result] - [https://moztrap.mozilla.org/results/cases/?&pagenumber=1&pagesize=100&sortfield=created_on&sortdirection=desc&filter-resultstatus=failed&filter-run=6552 30 fail], 1216 pass, 100% complete
	** [http://mzl.la/18T1AMy Bug list]		** [http://mzl.la/18T1AMy Bug list]

	* Bluetooth PTS/V2.2 Flame-KK <span style="color:#000000; background:#~~FFFF00~~"> Done </span>		* Bluetooth PTS/V2.2 Flame-KK <span style="color:#000000; background:#008000"> Done </span>
	** [https://bugzilla.mozilla.org/showdependencytree.cgi?id=1139349&hide_resolved=1 Bug Meta dependency tree] - 0 open bugs		** [https://bugzilla.mozilla.org/showdependencytree.cgi?id=1139349&hide_resolved=1 Bug Meta dependency tree] - 0 open bugs

Echang: /* V2.2 Testing Scope */

2015-04-27T10:28:12Z

V2.2 Testing Scope

← Older revision		Revision as of 10:28, 27 April 2015
Line 17:		Line 17:
	* [[B2G/QA/2.2#V2.2_QA_Owners\|B2G V2.2 Overview Page]] for more info.		* [[B2G/QA/2.2#V2.2_QA_Owners\|B2G V2.2 Overview Page]] for more info.

	* NEW [http://mzl.la/1B95Yi8 blocking-b2g=2.2+] 49 bugs		* Status NEW: [http://mzl.la/1B95Yi8 blocking-b2g=2.2+] 49 bugs
	<bugzilla>		<bugzilla>
	{		{

Echang: /* V2.2 Milestone */

2015-04-27T10:27:08Z

V2.2 Milestone

← Older revision		Revision as of 10:27, 27 April 2015
Line 12:		Line 12:
	* <strike>2015/04/06 V2.2 FC</strike>		* <strike>2015/04/06 V2.2 FC</strike>
	* 2015/02/23 V2.2 FL		* 2015/02/23 V2.2 FL

	* QA Shared calendar
	** You can add it to your Google Calendar view by clicking on this [https://www.google.com/calendar/embed?src=bW96aWxsYS5jb21fMmQzNTMxMzYzMTMzMzUzNTM5MmQzNjMxMzBAcmVzb3VyY2UuY2FsZW5kYXIuZ29vZ2xlLmNvbQ link] and then clicking on the small blue "+ Google Calendar" button at the bottom, right hand of the screen.
	** If you are signed in with your @mozilla.com account, you can also access it this way:
	*** In Google Calendar, you should go to Other calendars > Browse Interesting Calendars > More > Resources for mozilla.com > Public (9) > Public - FxOS QA Schedule > Subscribe

	== V2.2 Testing Scope ==		== V2.2 Testing Scope ==

Echang: /* 2nd Run (2015/04/14) */

2015-04-27T09:59:37Z

2nd Run (2015/04/14)

← Older revision		Revision as of 09:59, 27 April 2015
Line 56:		Line 56:
	==== Regression Testing ====		==== Regression Testing ====
	== 2nd Run (2015/04/14) ==		== 2nd Run (2015/04/14) ==
	* NEXUS-5-L (~~80% done~~)		* '''NEXUS-5-L 2nd Run (focus on essential functionality)'''
	** http://mzl.la/1D76gXG		** Test cases http://mzl.la/1D76gXG
	* Result
	* 1076(total), 1049(passed), 6(failed), 21(invalid)
	* 0.005% fail rate

	* FLAME-KK (~~done~~)		* '''Result'''
	** http://mzl.la/1aKYIC8		** 1119(total), 1090(passed), 8(failed), 21(invalid)
	* Result		** 0.007% fail rate
	* ~~342~~(total), 334(passed), 5(failed), 3(invalid)
	* 0.01% fail rate		* '''bugs'''
			** Total 5
			** http://mzl.la/1cYZZau (Bugs related to software home button, privacy or ui polish will be skipped for now)

			* '''FLAME-KK 2nd Run (focus on dsds and gaia apps)'''
			** Test cases http://mzl.la/1aKYIC8

			* '''Result'''
			** 342(total), 334(passed), 5(failed), 3(invalid)
			** 0.01% fail rate

			* '''bugs'''
			** Total 5
			** http://mzl.la/1GifEOz

	== 1st Run (2015/03/03) ==		== 1st Run (2015/03/03) ==

Echang: /* Stability */

2015-04-27T09:52:22Z

Stability

← Older revision		Revision as of 09:52, 27 April 2015
Line 183:		Line 183:
	- Result		- Result
	* NEXUS-5-L 5.1		* NEXUS-5-L 5.1
	** Current run - NEXUS-5-L 5.1 * 4: 77 hrs and counting''' = [55+93(running)+93(running) +67(running)] / 4 = 77		** Current run - NEXUS-5-L 5.1 *4: '''38.5 hrs and counting''' = [55+93(running)+93(running) +67(running)] / (4 devices + 4 crashes) = 38.5
	** Current run - ZTE-L 5.1 * 2: Just started		** Current run - ZTE-L 5.1 * 2: Just started
	** Previous run - NEXUS-5-L 5.1 * 4: MTBF 34.42 hrs = (41.3 + 53.2 + 43.8 + 6.1)/4		** Previous run - NEXUS-5-L 5.1 *4: MTBF 34.42 hrs = (41.3 + 53.2 + 43.8 + 6.1)/4
	** NEXUS-5-L 5.1 Rest time: 10%		** NEXUS-5-L 5.1 Rest time: 10%
	** [script issue] OSError: [Errno 24] Too many open files.		** [script issue] OSError: [Errno 24] Too many open files.

Echang: /* Stability */

2015-04-27T09:31:05Z

Stability

← Older revision		Revision as of 09:31, 27 April 2015
Line 183:		Line 183:
	- Result		- Result
	* NEXUS-5-L 5.1		* NEXUS-5-L 5.1
	** Current run - NEXUS-5-L 5.1 * 4: 77 hrs = [55+93(running)+93(running) +67(running)] / 4 = 77		** Current run - NEXUS-5-L 5.1 * 4: 77 hrs and counting''' = [55+93(running)+93(running) +67(running)] / 4 = 77
	** Current run - ZTE-L 5.1 * 2: Just started		** Current run - ZTE-L 5.1 * 2: Just started
	** Previous run - NEXUS-5-L 5.1 * 4: MTBF 34.42 hrs = (41.3 + 53.2 + 43.8 + 6.1)/4		** Previous run - NEXUS-5-L 5.1 * 4: MTBF 34.42 hrs = (41.3 + 53.2 + 43.8 + 6.1)/4
Line 195:		Line 195:
	** Display DIM		** Display DIM
	** Rest time with dummy case (30 sec)		** Rest time with dummy case (30 sec)
	** Running tests
	** 14 case http://bit.ly/1aWkJyY
	** FLAME *9		** FLAME *9