Mobile/Fennec Maemkit

From MozillaWiki
Jump to: navigation, search

Maemkit Toolset

NEW: Download the initial drop of [maemkit-chunked].

Until we get a lot of changes backported into the build system (makefiles, test running scripts, etc...) we will have a temporary test framework that wraps around the build system and test scripts to run all tests on maemo. We will call this the maemkit.

Here are a list of things the maemkit needs to do:


  • Package the tests
    • Ted has resolved this with 'make package-tests'
  • copy to device/unpack (leave manual for now)
  • run tests in a chunked method
    • mochitest - run_tests_chunked.py
      • test 1 directory at a time
      • splits known large directories into smaller chunks
    • chrometest - run_chrome_chunked.py
      • make _tests/testing/chrome.bak
      • copies 1 directory at a time from chrome.bak to chrome
    • reftest - reftest_chunked.py
      • parse manifest, run 1 dir at a time
      • for large dir, split into multiple manifests
    • crashtest - reftest_chunked.py
      • parse manifest, run 1 dir at a time
  • concat logs into a common log format
    • mochitest - run script to parse each chunk and cat together with summary
    • chrometest - run script to parse each chunk and cat together with summary
    • reftest - run script to remove file:///<rootdir> tags and cat together
    • crashtest - cat log files together
  • copy logs off device (leave manual for now)
  • report results with new failures highlighted
    • a system to keep track of known failures (json data provider)
    • a system to show results of tests including historical data


I propose writing a master script and 1 website to deal with this:

  • maemkit-chunked.py - initial draft [here]
    • run a test (or all) as chunked
    • reads a config file to determine
      • what directories to ignore, split
      • how many directories/tests to run
      • which directories/tests to run (if we are running in parallel)
    • create a log file that is compatible for comparison purposes
  • maemkit-viewer website
    • provider on known failures
    • provides quick summary of tests
    • provides in depth view of tests
    • provides access to download logs
    • TODO: determine a method for delivering failed test cases and related info to viewer. ctalbert suggests using json and mikeal has a module written for mozmill we might be able to use