|
|
| (2 intermediate revisions by the same user not shown) |
| Line 7: |
Line 7: |
| Below is a table outlining what is needed and where we are. Below the table is a log of actions which is intended for people to see what has been done and when if they want more details about a specific item. | | Below is a table outlining what is needed and where we are. Below the table is a log of actions which is intended for people to see what has been done and when if they want more details about a specific item. |
|
| |
|
| === Maemkit Toolset ===
| |
|
| |
|
| NEW: Download the initial drop of [[http://people.mozilla.com/~jmaher/fennec/maemkit/ maemkit-chunked]].
| |
|
| |
| Until we get a lot of changes backported into the build system (makefiles, test running scripts, etc...) we will have a temporary test framework that wraps around the build system and test scripts to run all tests on maemo. We will call this the maemkit.
| |
|
| |
| Here are a list of things the maemkit needs to do:
| |
|
| |
| * Package the tests
| |
| ** mochitest: tar -chf $(xr_objdir)/_tests/ $(objdir)/_profile/ tests.tar
| |
| ** chrometest: same as mochitest
| |
| ** reftest: run reftest_extract.py
| |
| ** crashtests: run crashtest_extract.py
| |
| * copy to device/unpack (leave manual for now)
| |
| * run tests in a chunked method
| |
| ** mochitest - run_tests_chunked.py
| |
| *** test 1 directory at a time
| |
| ***splits known large directories into smaller chunks
| |
| ** chrometest - run_chrome_chunked.py
| |
| *** make _tests/testing/chrome.bak
| |
| *** copies 1 directory at a time from chrome.bak to chrome
| |
| ** reftest - reftest_chunked.py
| |
| *** parse manifest, run 1 dir at a time
| |
| *** for large dir, split into multiple manifests
| |
| ** crashtest - reftest_chunked.py
| |
| *** parse manifest, run 1 dir at a time
| |
| * concat logs into a common log format
| |
| ** mochitest - run script to parse each chunk and cat together with summary
| |
| ** chrometest - run script to parse each chunk and cat together with summary
| |
| ** reftest - run script to remove file:///<rootdir> tags and cat together
| |
| ** crashtest - cat log files together
| |
| * copy logs off device (leave manual for now)
| |
| * report results with new failures highlighted
| |
| ** a system to keep track of known failures (json data provider)
| |
| ** a system to show results of tests including historical data
| |
|
| |
|
| |
| I propose writing 2 master scripts and 1 website to deal with this:
| |
| * maemkit-packager.py
| |
| ** packages all (or one type) the test files from the src and objdir trees
| |
| ** creates a .tar file that we can copy
| |
| * maemkit-chunked.py - initial draft [[http://people.mozilla.com/~jmaher/fennec/maemkit/ here]]
| |
| ** run a test (or all) as chunked
| |
| ** reads a config file to determine
| |
| *** what directories to ignore, split
| |
| *** how many directories/tests to run
| |
| *** which directories/tests to run (if we are running in parallel)
| |
| ** create a log file that is compatible for comparison purposes
| |
| * maemkit-viewer website
| |
| ** provider on known failures
| |
| ** provides quick summary of tests
| |
| ** provides in depth view of tests
| |
| ** provides access to download logs
| |
| ** TODO: determine a method for delivering failed test cases and related info to viewer. ctalbert suggests using json and mikeal has a module written for mozmill we might be able to use
| |
|
| |
|
| |
| NOTE: Ted is working {{bug|421611}} to package mochitest up with make package. Once this is finished, we can look into similar ways for reftests and xpcshell.
| |
|
| |
|
| === Nokia Status Table === | | === Nokia Status Table === |