QA/Fennec: Difference between revisions

From MozillaWiki
< QA
Jump to navigation Jump to search
(→‎How you can contact us: Use canonical link)
 
(28 intermediate revisions by 7 users not shown)
Line 1: Line 1:
Home of those who want to Propel Fennec Quality!
= Overview =
The team focuses on testing current and upcoming releases of Firefox on Android. Some of our responsibilities include qualifying builds prior to a maintenance or milestone release. Some of the responsibilities include, but are not limited to:


== Milestones ==
* Verifying bugs using a triage strategy
'''Current Releases'''
* Running basic functional tests and smoke tests; manual and automated
* Native: [https://docs.google.com/spreadsheet/ccc?key=0AocUyLHteCtSdHQ5Q2tIZVhMT3NNY0lPYzhHT2MyZXc&hl=en_US#gid=0 Native Testplan]
* Running our manual test suites on localized builds
* 10.0: [https://wiki.mozilla.org/QA/Fennec/Milestones/10.0 Testplan]
* Making sure web content is viewable, acceptable and ready for consumption in Firefox for Android


{{hidden
In addition to qualifying builds prior to a release, we perform ongoing tasks on the development branches to ensure no major problems get uplifted as we merge changes to the next branch. For example, on a daily basis we:
|Past Releases
|
* 9.0: [https://wiki.mozilla.org/QA/Fennec/Milestones/9.0 Testplan]
* 8.0: [https://wiki.mozilla.org/QA/Fennec/Milestones/8.0 Testplan]
* 7.0: [https://wiki.mozilla.org/QA/Fennec/Milestones/7.0 Testplan]
* 6.0: [https://wiki.mozilla.org/QA/Fennec/Milestones/6.0 Testplan]
* 5.0: [https://wiki.mozilla.org/QA/Fennec/Milestones/5.0 Testplan]
* 2.0: [https://wiki.mozilla.org/QA/Fennec/Milestones/2.0 Testplan]
* 1.1: [https://wiki.mozilla.org/QA/Fennec/Milestones/1.1 Testplan]
* 1.0: [https://wiki.mozilla.org/QA/Fennec1.0/TestPlan Testplan]
|headerstyle=background:#dddddd
}}


== Planning ==
* Confirm new unconfirmed bugs
* [https://wiki.mozilla.org/QA/Fennec/Planning/2011 2011]
* Verify bugs on development branches like Nightly and Beta
* <s>[https://wiki.mozilla.org/QA/Fennec/Planning/2010 2010]</s>
* Engage community and new volunteers in helping improve Firefox through test day events
* Interact with developers to help them test the features they implement
* Perform exploratory testing of new features on the Nightly branch
* Monitor [http://crash-stats.mozilla.com/ crash-stats] to assess stability
* File and track new crasher bugs as they appear in crash-stats
* Manage web compatibility testing through community involvement or crowdsourcing companies that help in testing different Android device configurations and top-site testing


== What We Do ==
All of which help assess the quality of individual new features as well as the Firefox version as a whole.


= Team Details =
The team is lead by both Kevin Brosnan and Aaron Train who coordinate testing of their respective areas among several MoCo employees and community volunteers. Traditionally new features are tested by several MoCo QA employees. Anyone is welcome and encouraged to participate.


=== Testing Strategy  ===
* Lead - [mailto:kbrosnan@mozilla.com Kevin Brosnan (:kbrosnan)]
; Manual
* Lead - [mailto:atrain@mozilla.com Aaron Train (:aaronmt)]
*[[QA/Fennec/Waverley]] on the latest [http://nightly.mozilla.org Nightly build] on the current development branch and trunk
*Verification of overnight resolved bugs on the current development branch
*Run BFT(s)/Focused testing around areas that area affected by resolved bugs
*Weekly triage of [http://bit.ly/qbtF9O new and unconfirmed bugs]
*[http://quality.mozilla.org/ Bi-Monthly Testdays]


; Automation
== Team Members and Assignments ==
* [[Mobile/Fennec/Android#Testing|Android Testing]]
* [https://wiki.mozilla.org/User:MarkFinkle/RunningTests Browser Chrome, Reftests, Crashtests, Talos Performance Tests]
 
; Unique Areas of Testing:
{| class="fullwidth-table"
{| class="fullwidth-table"
|-
| style="background:#EFEFEF; width:30%" | '''Name'''
| style="background:#EFEFEF; width:33%" | '''Project'''  
| style="background:#EFEFEF; width:30%" | '''Contact'''
| style="background:#EFEFEF; width:66%" | '''Description'''
| style="background:#EFEFEF; width:10%" | '''Availability'''
|- valign="top"
| style="background:#EFEFEF; width:30%" | '''Project Assignments'''
| [https://spreadsheets.google.com/ccc?key=0AhE7m4JB2j6tdEJFZ3NoZkRoRTM3TTRpQll3UjZLYnc&hl=en&authkey=CPiPp9IK#gid=4 Device Compatibility Test Plan]
|-
| Tracks Tests and Full device list of compatibility with Mobile Firefox
|Aaron Train
|-
|[mailto:atrain@mozilla.com atrain@mozilla.com]
| [[QA/Fennec/WebsiteCompatibility|Website Compatibility Test Plan]]
|MoCo Employee (full time)
| Tracks top Website compatibility with Mobile Firefox
|Lead - New features testing coordination.
|-
|-
| [https://etherpad.mozilla.org/fennec-perf-ts-take2 Performance Benchmarking Test Plan]  
|Kevin Brosnan
| Tracking performance benchmarking numbers for Startup times and Page Loads
|[mailto:kbrosnan@mozilla.com kbrosnan@mozilla.com]
|-
|MoCo Employee (full time)
| [https://wiki.mozilla.org/QA/Browser_Technologies/Mobile_Unit_Testing_Addon Mobile Addon for Unit Testing Automation]
|Lead - New features testing coordination.
| Project to run unit tests via a test-harness addon and disperse via crowdsourcing techniques
|-
|}
|}
 
=== Release Testing ===
* [https://wiki.mozilla.org/Mobile/Releases/Release_Checklist Mobile Release Checklist]
 
=== Fennec Crash Reporting ===
 
== What We Use ==
 
 
=== Tools ===
; Litmus
* [http://litmus.mozilla.org/ Fennec Test Runs]
 
; Device Anywhere
* [http://wiki.mozilla.org/QA/Fennec/DeviceAnywhere DeviceAnywhere]
 
; Performance Graphs
* [http://graphs.mozilla.org/ Performance Graphs]
 
=== Resources ===
; Device Anywhere Tutorial
* [http://wiki.mozilla.org/QA/Fennec/DeviceAnywhere Device Anywhere Tutorial]
 
; Mobile (HG) Repository
* [http://hg.mozilla.org/mozilla-central/ mozilla-central]
 
; Mobile Source Code
* [http://mxr.mozilla.org/mozilla-central/source/mobile/ Fennec MXR Source Code] Web-Based Viewer
 
; Builds
* [http://nightly.mozilla.org/mobile.html Mobile Nightly Builds]
* [http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/ Hourly Tinderbox Builds]
* [https://l10n-stage-sj.mozilla.org/?tree=fennec10x Fennec L10n Dashboard]
 
== Where Do We Need Help ==
 
=== Test Tool Development ===
* Write automated [https://wiki.mozilla.org/Mobile/Fennec_TestDev#Areas_to_focus_on testcases]
* Port [https://wiki.mozilla.org/Mobile/Fennec_Automation Automation] Frameworks to WinCE
* Find, Fix, Verify [https://wiki.mozilla.org/Mobile/Fennec_TestDev#Other_TestDev_related_todo.27s UnitTest] Failures
 
=== Execution ===
* Triage [https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced;bug_status=UNCONFIRMED;product=Fennec Unconfirmed] Fennec Bugs
* Create Litmus testcases for bugs flagged as [https://bugzilla.mozilla.org/buglist.cgi?cmdtype=runnamed&namedcmd=in-litmus%3F%20Mobile in-litmus?]
** [https://wiki.mozilla.org/QA/Execution/Litmus/Ownership Fennec Component Ownership in Litmus]
* [https://wiki.mozilla.org/QA/TestDay/ TestDay] Promotions within Mobile Community Channels
* Write [http://wiki.mozilla.org/QA/Fennec/TechDocQueue technical documentation] to perform quality execution
 
=== Performance Benchmarking ===
* Scratch pad of [http://etherpad.mozilla.com:9000/3nS0MfR7yx notes]
* [[QA/Fennec/PerformanceBenchmark|Performance Testplan]]
 
=== Website Compatibility Testing ===
* Scratch pad of [http://etherpad.mozilla.com:9000/3nS0MfR7yx notes]
* [[QA/Fennec/WebsiteCompatibility|Web Compatibility Testplan]]


=== Sync Performance Instrumentation ===
=Community Contribution=
Fennec sync continues to cause performance issues on first time syncing with very large amounts of data.   How you can help, is to provide sync logging through a custom extension, and submit your data to the Mozilla Services team.  Detailed instructions can be found [[QA/Fennec/Sync_Performance_Collection|here]].
'''Anyone can participate.''' There are several ways in which '''YOU''' can participate:
<p>
* File bugs
More historical information on the project:
* Triage bugs (confirm existing bugs and assign them to the right buckets)
* Philikon's [https://wiki.mozilla.org/User:Philikon/SyncInstrumentation Sync Instrumentation Research]
* Help test new features
* Blog on [http://philikon.wordpress.com/2011/03/09/javascript-perf-avoid-creating-objects-2/ profiling Javascript Objects]
* Write test cases
* Plan new features testing
* Help others who want to get involved.


== How You Can Contact Us ==
=== Download Firefox for Android ===
If you're interested in helping out with anything below (or want to help with something else), please contact [http://quality.mozilla.org/user/21/contact jmaher]  or [http://quality.mozilla.org/user/1342/contact aakashd]. If you want to talk to the Mozilla Mobile Community, you can find us in our [http://widget.mibbit.com/?settings=1b10107157e79b08f2bf99a11f521973&server=irc.mozilla.org&channel=%23mobile IRC Channel].  
The easiest and most simplest way to help us by using Firefox for Android. One can download Firefox for Android on [https://play.google.com/store/apps/details?id=org.mozilla.firefox Google Play] and installing it on your Android device.


* Mailing Lists
== How you can contact us ==
** mobile@mozilla.com
If you're interested in contributing or have any other question, get in touch on [[Matrix]]:
* Newsgroups
* [https://chat.mozilla.org/#/room/#firefox-android:mozilla.org #firefox-android] on [https://chat.mozilla.org/#/room/#firefox-android:mozilla.org chat.mozilla.org]
** [http://groups.google.com/group/mozilla.dev.platforms.mobile/topics mozilla.dev.platforms.mobile]
** [http://groups.google.com/group/mozilla.dev.quality/topics mozilla.dev.quality]
* 3rd Party Testing (Waverley)
** [https://wiki.mozilla.org/QA/Fennec/Waverley Execution testplan] for Smoketests & BFTs

Latest revision as of 14:54, 14 April 2026

Overview

The team focuses on testing current and upcoming releases of Firefox on Android. Some of our responsibilities include qualifying builds prior to a maintenance or milestone release. Some of the responsibilities include, but are not limited to:

  • Verifying bugs using a triage strategy
  • Running basic functional tests and smoke tests; manual and automated
  • Running our manual test suites on localized builds
  • Making sure web content is viewable, acceptable and ready for consumption in Firefox for Android

In addition to qualifying builds prior to a release, we perform ongoing tasks on the development branches to ensure no major problems get uplifted as we merge changes to the next branch. For example, on a daily basis we:

  • Confirm new unconfirmed bugs
  • Verify bugs on development branches like Nightly and Beta
  • Engage community and new volunteers in helping improve Firefox through test day events
  • Interact with developers to help them test the features they implement
  • Perform exploratory testing of new features on the Nightly branch
  • Monitor crash-stats to assess stability
  • File and track new crasher bugs as they appear in crash-stats
  • Manage web compatibility testing through community involvement or crowdsourcing companies that help in testing different Android device configurations and top-site testing

All of which help assess the quality of individual new features as well as the Firefox version as a whole.

Team Details

The team is lead by both Kevin Brosnan and Aaron Train who coordinate testing of their respective areas among several MoCo employees and community volunteers. Traditionally new features are tested by several MoCo QA employees. Anyone is welcome and encouraged to participate.

Team Members and Assignments

Name Contact Availability Project Assignments
Aaron Train atrain@mozilla.com MoCo Employee (full time) Lead - New features testing coordination.
Kevin Brosnan kbrosnan@mozilla.com MoCo Employee (full time) Lead - New features testing coordination.

Community Contribution

Anyone can participate. There are several ways in which YOU can participate:

  • File bugs
  • Triage bugs (confirm existing bugs and assign them to the right buckets)
  • Help test new features
  • Write test cases
  • Plan new features testing
  • Help others who want to get involved.

Download Firefox for Android

The easiest and most simplest way to help us by using Firefox for Android. One can download Firefox for Android on Google Play and installing it on your Android device.

How you can contact us

If you're interested in contributing or have any other question, get in touch on Matrix:

Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Fennec&oldid=1257001"