Overview

The MTBF tests are a suite of Firefox OS tests built on top of the Gaiatest (Gaia UI Tests) Framework. The tests run on real Firefox OS devices, and use Marionette to drive the device's UI.

Mean time between failures (MTBF) is the predicted elapsed time between inherent failures of a system during operation. MTBF can be calculated as the arithmetic mean (average) time between failures of a system. The MTBF is typically part of a model that assumes the failed system is immediately repaired (mean time to repair, or MTTR), as a part of a renewal process. This is in contrast to the mean time to failure (MTTF), which measures average time to failures with the modeling assumption that the failed system is not repaired (infinite repair time).

Assuming T(i) is ith running time between two failures, MTBF will be SUM ( T ( n ) ) / n

Below is information for MTBF in Mozilla

Pass Criteria

Currently we set 100 hours as our pass criteria. Crash, hang are regarded as failure.

Test Scope

The scope of V2.1 is still under determination. We might still change it slightly.

Test Suite

• Keyboard - 1. number keyboard 2. add contact 3. email keyboard
• Music - play 10 sec music in song view
• Message - send a message to itself
• FMRadio - play and see radio is enable
• Dialer - make a call from call log
• Cards View - Open two apps and kill one
• Camera - take two pictures
• Settings - turn on and turn off wifi
• Browser - 1. browse a website 2. add a bookmark 3. browse from bookmark
• Video - play a very short video

Backlog

• More test suites (on-going)
  • airplane mode on/off
• Crash info detection
• Report enhancement
• Integrating mozbase b2g related modules
• Datazilla integration

Current Bugs

MTBF Meta Bug

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);

MTBF Marionette Meta Bug

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);

MTBF B2G Meta Bug

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);

MTBF-2015Q3

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);

Documentation

MTBF Reports History

MTBF Trend

Moztrap MTBF Test Cases

References

• Getting Started
  • Javascript
    • http://www.codecademy.com/tracks/javascript (online course)
    • http://bonsaiden.github.io/JavaScript-Garden/ (good practices)
    • http://eloquentjavascript.net/ (book - accessible as HTML)
    • https://developer.mozilla.org/en-US/docs/Web/JavaScript (reference)
  • Python
    • http://www.codecademy.com/tracks/python
    • https://wiki.python.org/moin/BeginnersGuide
  • HTML / CSS
    • http://www.codecademy.com/tracks/web