Mobile/Fennec Automation: Difference between revisions

Jump to navigation Jump to search

Mobile/Fennec Automation (view source)

Revision as of 15:20, 18 June 2009

2,357 bytes removed ,  18 June 2009
→‎Maemkit Toolset
 
Line 7: Line 7:
Below is a table outlining what is needed and where we are.  Below the table is a log of actions which is intended for people to see what has been done and when if they want more details about a specific item.
Below is a table outlining what is needed and where we are.  Below the table is a log of actions which is intended for people to see what has been done and when if they want more details about a specific item.


=== Maemkit Toolset ===


NEW: Download the initial drop of [[http://hg.mozilla.org/qa/maemkit/file/527dc910d556/ maemkit-chunked]].
Until we get a lot of changes backported into the build system (makefiles, test running scripts, etc...) we will have a temporary test framework that wraps around the build system and test scripts to run all tests on maemo.  We will call this the maemkit.
Here are a list of things the maemkit needs to do:
* Package the tests
** Ted has resolved this with 'make package-tests'
* copy to device/unpack (leave manual for now)
* run tests in a chunked method
** mochitest - run_tests_chunked.py
*** test 1 directory at a time
***splits known large directories into smaller chunks
** chrometest - run_chrome_chunked.py
*** make _tests/testing/chrome.bak
*** copies 1 directory at a time from chrome.bak to chrome
** reftest - reftest_chunked.py
*** parse manifest, run 1 dir at a time
*** for large dir, split into multiple manifests
** crashtest - reftest_chunked.py
*** parse manifest, run 1 dir at a time
* concat logs into a common log format
** mochitest - run script to parse each chunk and cat together with summary
** chrometest - run script to parse each chunk and cat together with summary
** reftest - run script to remove file:///<rootdir> tags and cat together
** crashtest - cat log files together
* copy logs off device (leave manual for now)
* report results with new failures highlighted
** a system to keep track of known failures (json data provider)
** a system to show results of tests including historical data
I propose writing a master script and 1 website to deal with this:
* maemkit-chunked.py - initial draft [[http://hg.mozilla.org/qa/maemkit/file/527dc910d556/ here]]
** run a test (or all) as chunked
** reads a config file to determine
*** what directories to ignore, split
*** how many directories/tests to run
*** which directories/tests to run (if we are running in parallel)
** create a log file that is compatible for comparison purposes
* maemkit-viewer website
** provider on known failures
** provides quick summary of tests
** provides in depth view of tests
** provides access to download logs
** TODO: determine a method for delivering failed test cases and related info to viewer.  ctalbert suggests using json and mikeal has a module written for mozmill we might be able to use


=== Nokia Status Table ===
=== Nokia Status Table ===
Jmaher
Confirmed users
3,376

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/150381"

Navigation menu