Mobile/Fennec Unittests/green

overview

This is a list of tests which I found to fail on both the n900 and n810 over repeated runs. I believe making these tests pass (either by fixing the tests or ignoring the tests and writing new ones if necessary) will get us to a good position where we can start seeing green.

Until reftest/crashtest/xpcshell are green we will continue to ignore the mochitest stuff.

mochitest

List of [bugs on file] for failing unitests

Just to jump start our mochitest stuff, I have created a spreadsheet of the different directories in mochitest-plain that have tests: https://spreadsheets.google.com/ccc?key=0ArS97F99-BEZdGZtTkxFSFBaTkNHbkdRVlFXbmNQZWc&authkey=CJiu1soC&hl=en#gid=0

Over the next few weeks, mw22 will lead the effort to:

• discover failing test files (verified and reproduced)
• if the test case is not for Fennec we will disable it via makefile [example]
• if the test case is failing due to a preference, we will make a decision to disable the test by checking in the test case for the specific preference and value, change the preference for mobile testing, or file and track a bug to fix broken functionality in Fennec
• if the test case is failing due to other issues, we will attempt to categorize the issue as best as possible (without spending a lot of time), file a bug and move on

As a note we are only focusing on:

• fennec 2.0 gtk builds (on the n900) with browser.tabs.remote=false. There are too many test files that need to be fixed
• virtual keyboard turned off (settings->text_input-> uncheck use virtual keyboard)
• testing on the device via ssh
• future testing will include verifying the same results with remote testing
• future testing will include identifying e10s failing cases and verifying they are not fennec specific failures.

Some directories need to be run in chunks or the browser will crash (OOM) as a result of the large memory usage xpcshell consumes. Here is an example of a simple shell script to chunk up dom/tests/mochitest/dom-level1-core.

for i in 1 2 3 4 5 6 7 8
do
  python runtests.py --appname=../fennec/fennec --certificate-path=../certs --utility-path=../bin --autorun --log-file=level1-$i.log --close-when-done --test-path=dom/tests/mochitest/dom-level1-core --total-chunks=8 --this-chunk=$i
done

reftest

Here are some guides for running the tests:

• [maemo]
• [winmo]

crashtest

Here are some guides for running the tests:

• [maemo]
• [winmo]

xpcshell

Here are some guides for running the tests:

• [maemo]
• [winmo]