Mobile/Fennec Unittests/green

From MozillaWiki
Jump to: navigation, search

overview

This is a list of tests which I found to fail on both the n900 and n810 over repeated runs. I believe making these tests pass (either by fixing the tests or ignoring the tests and writing new ones if necessary) will get us to a good position where we can start seeing green.

Until reftest/crashtest/xpcshell are green we will continue to ignore the mochitest stuff.

mochitest

List of [bugs on file] for failing unitests

Just to jump start our mochitest stuff, I have created a spreadsheet of the different directories in mochitest-plain that have tests: https://spreadsheets.google.com/ccc?key=0ArS97F99-BEZdGZtTkxFSFBaTkNHbkdRVlFXbmNQZWc&authkey=CJiu1soC&hl=en#gid=0

Over the next few weeks, mw22 will lead the effort to:

  • discover failing test files (verified and reproduced)
  • if the test case is not for Fennec we will disable it via makefile [example]
  • if the test case is failing due to a preference, we will make a decision to disable the test by checking in the test case for the specific preference and value, change the preference for mobile testing, or file and track a bug to fix broken functionality in Fennec
  • if the test case is failing due to other issues, we will attempt to categorize the issue as best as possible (without spending a lot of time), file a bug and move on

As a note we are only focusing on:

  • fennec 2.0 gtk builds (on the n900) with browser.tabs.remote=false. There are too many test files that need to be fixed
  • virtual keyboard turned off (settings->text_input-> uncheck use virtual keyboard)
  • testing on the device via ssh
  • future testing will include verifying the same results with remote testing
  • future testing will include identifying e10s failing cases and verifying they are not fennec specific failures.


Some directories need to be run in chunks or the browser will crash (OOM) as a result of the large memory usage xpcshell consumes. Here is an example of a simple shell script to chunk up dom/tests/mochitest/dom-level1-core.

for i in 1 2 3 4 5 6 7 8
do
  python runtests.py --appname=../fennec/fennec --certificate-path=../certs --utility-path=../bin --autorun --log-file=level1-$i.log --close-when-done --test-path=dom/tests/mochitest/dom-level1-core --total-chunks=8 --this-chunk=$i
done

reftest

Here are some guides for running the tests:

test status bug
tests/modules/libpr0n/test/reftest/colordepth.html
tests/modules/libpr0n/test/reftest/pngsuite-ancillary/ccwn2c08.png bug 471917
tests/modules/libpr0n/test/reftest/pngsuite-ancillary/ccwn3p08.png bug 471917
tests/layout/reftests/bugs/134706-3-left-scrollframe.html pass with no-scrollbar bug 556087
tests/layout/reftests/bugs/134706-3-right-scrollframe.html pass with no-scrollbar bug 556087
tests/layout/reftests/bugs/240536-resizer-ltr.xul This is a nativetheme issue. It depends on the resize handle being rendered.
tests/layout/reftests/bugs/300691-1a.html spellcheck squiggle bug 556076
tests/layout/reftests/bugs/300691-1b.html spellcheck squiggle bug 556076
tests/layout/reftests/bugs/300691-1c.html spellcheck squiggle bug 556076
tests/layout/reftests/bugs/300691-1d.html spellcheck squiggle bug 556076
tests/layout/reftests/bugs/300691-1e.html spellcheck squiggle bug 556076
tests/layout/reftests/bugs/300691-1f.html spellcheck squiggle bug 556076
tests/layout/reftests/bugs/301726-2.html round2
tests/layout/reftests/bugs/338251-p-oh.html pass with no-scrollbar bug 556087
tests/layout/reftests/bugs/362901-1.html pass with no-scrollbar bug 556087
tests/layout/reftests/bugs/367247-s-auto.html round2
tests/layout/reftests/bugs/367247-l-hidden.html round2
tests/layout/reftests/bugs/371561-1.html round2
tests/layout/reftests/bugs/376532-2.html round2
tests/layout/reftests/bugs/376532-3.html round2
tests/layout/reftests/bugs/379461-3-container-xhtml.html pass with no-scrollbar bug 556087
tests/layout/reftests/bugs/379461-3-container-xhtml.html (!=) round2
tests/layout/reftests/bugs/385569-1a.html pass with no-scrollbar bug 556087
tests/layout/reftests/bugs/385569-1b.html pass with no-scrollbar bug 556087
tests/layout/reftests/bugs/386470-1b.html pass with no-scrollbar bug 556087
tests/layout/reftests/bugs/423823-1.html
tests/layout/reftests/bugs/424074-1-ref2.xul
tests/layout/reftests/bugs/440112.html pass with no-scrollbar bug 556087
tests/layout/reftests/bugs/466395-1.html pass with no-scrollbar bug 556087
tests/layout/reftests/bugs/466395-2.html pass with no-scrollbar bug 556087
tests/layout/reftests/bugs/467084-1.html pass with no-scrollbar bug 556087
tests/layout/reftests/bugs/474417-1.html round2
tests/layout/reftests/bugs/491180-1.html bug 548826
tests/layout/reftests/bugs/491180-2.html bug 548826
tests/layout/reftests/bugs/495385-2h.html pass with no-scrollbar bug 556087
tests/layout/reftests/bugs/513318-2.xul round2
tests/layout/reftests/bugs/513318-3.xul round2
tests/layout/reftests/canvas/text-bidi-ltr-test.html round2
tests/layout/reftests/canvas/text-bidi-rtl-test.html round2
tests/layout/reftests/generated-content/table-parts-01.html round2
tests/layout/reftests/font-face/src-list-local-full.html bug 548865
tests/layout/reftests/font-face/src-list-local-full-quotes.html bug 548865
tests/layout/reftests/font-face/local-1.html bug 548865
tests/layout/reftests/margin-collapsing/block-overflow-2.html pass with no-scrollbar bug 556087
tests/layout/reftests/margin-collapsing/block-overflow-2.html pass with no-scrollbar bug 556087
tests/layout/reftests/native-theme/text-input-native.html expected fail (native widget) bug 471700
tests/layout/reftests/native-theme/textarea-native.html expected fail (native widget) bug 471700
tests/layout/reftests/native-theme/button-native.html expected fail (native widget) bug 471700
tests/layout/reftests/native-theme/combobox-native.html expected fail (native widget) bug 471700
tests/layout/reftests/native-theme/listbox-native.html expected fail (native widget) bug 471700
tests/layout/reftests/native-theme/radio-native.html expected fail (native widget) bug 471700
tests/layout/reftests/native-theme/checkbox-native.html expected fail (native widget) bug 471700
tests/layout/reftests/native-theme/box-shadow-styling.html expected fail (native widget) bug 471700
tests/layout/reftests/svg/sizing/inline--display-inline--01.xhtml left edge of box is broken bug 548648
tests/layout/reftests/svg/sizing/inline--display-inline-block--01.xhtml left edge of box is broken bug 548648
tests/layout/reftests/svg/sizing/inline--position-relative--01.xhtml left edge of box is broken bug 548648
tests/layout/reftests/svg/sizing/dynamic--inline-resize-cb-height.xhtml pass with no-scrollbar bug 556087
tests/layout/reftests/svg/sizing/dynamic--inline-resize-cb-width.xhtml pass with no-scrollbar bug 556087
tests/layout/reftests/svg/sizing/dynamic--inline-resize-window-width.xhtml marked as fails in m-c
tests/layout/reftests/svg/opacity-and-gradient-01.svg pass with no-scrollbar bug 556087
tests/layout/reftests/text/wordwrap-01.html pass with no-scrollbar bug 556087
tests/layout/reftests/transform/iframe-1.html pass with no-scrollbar bug 556087
tests/content/test/reftest/xml-stylesheet/lreas_selflink_dtd_id.svg marked as fails in m-c
tests/layout/xul/base/src/grid/reftests/scrollable-columns.xul pass with no-scrollbar bug 556087

crashtest

Here are some guides for running the tests:

Test Status Bugs
tests/layout/base/crashtests/311611-1.html FAIL
tests/layout/base/crashtests/500467-1.html HANG

xpcshell

Here are some guides for running the tests:

Test Status Bugs
tests/jsctypes-test/unit/test_jsctypes.js
tests/test_uriloader_exthandler/unit/test_handlerService.js
tests/test_docshell/unit/test_bug414201_jfif.js
tests/test_places/autocomplete/test_download_embed_bookmarks.js
tests/test_places/autocomplete/test_livemarks.js
tests/test_places/sync/test_expire_on_timed_flush.js
tests/test_places/sync/test_expire_on_timed_flush_not_whole_entry.js
tests/test_places/unit/test_000_frecency.js
tests/test_places/unit/test_408221.js
tests/test_places/unit/test_empty_tags.js HANGS
tests/test_places/unit/test_frecency.js
tests/test_places/unit/test_history_autocomplete_tags.js
tests/test_places/unit/test_migrateFrecency.js
tests/test_satchel/unit/test_autocomplete.js
tests/test_satchel/unit/test_expire.js
tests/test_extensionmanager/unit/test_bug299716.js
tests/test_extensionmanager/unit/test_bug371495.js
tests/test_extensionmanager/unit/test_bug396129.js
tests/test_extensionmanager/unit/test_bug428341.js
tests/test_extensionmanager/unit/test_bug463819.js
tests/test_extensionmanager/unit/test_bug470377_1.js
tests/test_extensionmanager/unit/test_bug470377_2.js
tests/test_necko/unit/test_file_protocol.js
tests/test_necko/unit/test_localstreams.js