New Hire Documentation for Firefox OS QA

This document was created as a supplement to the New Hire Document For QA

Tools and Processes

QA Lab

There is a QA lab in Mountain View on the second floor (right next to the kitchen) that has a variety of hardware that we test with. All members of the QA team automatically have badged access to the room, but if you are a new hire you may still need to request access.

Tinderbox Build

Triage

Bug triage is a process in which bugs are prioritized to be fixed. The bugs are sorted into categories like critical and non-critical to manage the inward flow of issues and to make sure the proper issues are being worked on at a given time.

Bugzilla

Mozilla’s own bug tracking software. We dogfood this application and use it to keep track of all of our issues and sometimes to manage projects as well. If the issue needs to be visible, it needs to be in Bugzilla.

• QA Wanted

  This is a flag used in Bugzilla to have QA perform a branch check (they will specify if they want to check something older than the latest stable).

  VerifyMe

  This flag is used in Bugzilla to have QA verify that a patch truly fixed the issue. These tasks are usually given to contractors.

  Nomination

  Action of flagging a bug in order to make it triaged. On Bugzilla, it's usually defined with the question mark like in blocking-b2g:2.2?.

  Blocker

  A blocker is a bug where critical functionality is broken to the point that we cannot release the product. The release is therefore "Blocked". On Buzilla, with the Firefox OS project, a blocker is defined with blocking-b2g and the + sign as in blocking-b2g:2.2+

TBPL

A tool, called Tinderboxpushlog, that correlates checkins with Buildbot results so that developers and sheriffs can quickly see the results of automatic builds and tests for specific change sets. TBPL has been replaced by Treeherder.

Gaia

The user interface layer of the Firefox OS Platform. Any part of the user interface that is drawn on the screen is done through the Gaia. Gaia is implemented entirely in HTML, CSS and JavaScript. It interfaces with the underlying operating system through open Web APIs, which are implemented through the Gecko layer.

Gecko

The Firefox OS application runtime. This is the layer that provides all of the support for HTML, CSS and JavaScript. It makes sure that those APIs work on every OS that Gecko supports. Gecko many layers including, but not limited to, a networking stack, graphics stack, layout engine, a JavaScript virtual machine and porting layers.

Gonk

Gonk is the lower level operating system of the Firefox OS platform. It consists of a Linux kernel (based on the Android Open Source Project) and a userspace hardware abstraction layer (HAL). Basically, one could say that Gonk is a very simple Linux distribution. Gonk is a porting target of Gecko.

B2G

B2G is short for Boot To Gecko which is a project a complete, standalone operating system in build for the Web. It is built entirely using web technologies (HTML/CSS/JavaScript).

Flame

This is the first generation reference device for Firefox OS.

Nexus 5

This is going to be the second generation reference device for Firefox OS. More to come.

Firefox OS

Mozilla’s offering for a mobile operating system. It is built completely on web technologies creating an innovative ecosystem for developers and gives users a choice.

Marionette

Marionette is an automation driver based on Selenium WebDriver for Mozilla's Gecko engine. It relies on 2 parts: the server (included in Gecko) and one client (either in Python or in JavaScript)

Buildbot

See New Hire Document For QA

Sheriffing

Tree Herder

Treeherder is the successor to TBPL. The purpose of Treeherder is to support longer term use-cases for managing and visualizing data and rewrite the existing application from scratch, so that it can scale appropriately in the future. It was created as a way to address some pain points with TBPL.

Gaia-try

A robot which is in charge of creating jobs on Treeherder against Pull requests on the Gaia repo.

Task Cluster

Mozilla Central

The main (a.k.a. trunk, a.k.a. master) branch of Firefox Desktop and Gecko.

Jenkins

The current on-device test runner. All the other tests are run on Treeherder.

Image Compare

Wiki

Wiki is the type of markup tool we use to post documentation. There is a quick tutorial that one can use to become familiar with it. For some Mozilla specific information about Wiki, see this tutorial.

Glossary

Kernel Panic

...On Device

This means that whatever action is being referenced is being performed on a physical Flame device.

...On Desktop

This means that whatever action is being referenced is being performed on B2G Desktop or Web IDE. This would not be appropriate for performance or hardware specific issues since those don't have access to the hardware of the phone.

G.I.P

Gaia Integration Python tests are the automation tests that are written in Python. We, as a group, are migrating away from the Python testing suite to use in favor of GIJ. This way, both the applications and the tests are using the same tech and will have better support.

G.I.J

Gaia Integration Javascript.

Shallow Flash

This process means you are flashing the Gecko and Gaia layers only. It assumes that all drivers and libraries for the hardware are in place and up to date.

Full Flash'

This process means that you are flashing all layers

QAnalyst

Regression Window

The timeframe between the last successful build and when it broke. The smaller the window, the better for assessing what went wrong.

Smoke Test

A test suite (automated and manual) run everyday to check the basic features of the product

Nightly

A build that happens daily, usually overnight.

Functional Team

Milestones (Release)

Feature Landing

Feature Complete

Dogfood

When you hear the term "dogfooding", it is the process of using a product that you created. So, in terms of Firefox OS QA, someone would use a Flame device as their own personal phone. Thus, they are testing the phone in the realest sense of the term.

Communication Channels

• IRC
  • #fxosqa
  • #fxos
• Telegram App (Download and install from https://telegram.org/)
  • Firefox OS QA
  • Quality Assurance & Mozillians

Important Mailing Lists

Sign up for these to stay up-to-date with FFOS!

• b2g-internal
• qa-b2g-internal
• CD_allstaff
• dev-fxos
• b2g-release-drivers
• fxosqa-report-watchlist

Reading List

• Testing Computer Software (Second Edition)