Auto-tools/Projects/B2G Automation: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
Line 13: Line 13:
# Timeout for WebAPI tests is too small; emulator callbacks can occasionally take 10-20s, leading to random timeouts.  Rob will fix.
# Timeout for WebAPI tests is too small; emulator callbacks can occasionally take 10-20s, leading to random timeouts.  Rob will fix.
# socket.timeout errors caused by actorPool error {{bug|859417}}
# socket.timeout errors caused by actorPool error {{bug|859417}}
# socket.timeout errors that occur at startup, before a new Gecko is copied
# socket.timeout errors that occur at startup in newSession, before a new Gecko is copied - see logs in {{bug|829551}}
# chrome tests occasionally can't find elements (timing or page loading error?) {{bug|859423}}
# chrome tests occasionally can't find elements (timing or page loading error?) {{bug|859423}}
# B2G crashes during WebAPI tests {{bug|859439}} - will be easier to handle once crash detection is working here
# B2G crashes during WebAPI tests {{bug|859439}} - will be easier to handle once crash detection is working here

Revision as of 16:37, 9 April 2013

Goal 2013 Q2

B2G Automation

Approach and Milestones

  • TODO: Add bugzilla bug queries to this by attaching proper whiteboard flags to the relevant bugs that are filed.

Milestone 1 - 2 weeks (tentative dates: April 1 - 12)

  • [jgriffin] Get Gaia Unit tests running in TBPL (blocked by bug 855049)
  • [jgriffin] [DONE] Triage all existing B2G failures and identify a concrete set of problems that need to be addressed
  1. test_click errors bug 838607 - dburns looking at it.
  2. reftest/crashtest crash bug 818103 - assigned to marshall_law.
  3. exception in window_manager.js bug 851666 - asked David Scravaglieri for help
  4. Timeout for WebAPI tests is too small; emulator callbacks can occasionally take 10-20s, leading to random timeouts. Rob will fix.
  5. socket.timeout errors caused by actorPool error bug 859417
  6. socket.timeout errors that occur at startup in newSession, before a new Gecko is copied - see logs in bug 829551
  7. chrome tests occasionally can't find elements (timing or page loading error?) bug 859423
  8. B2G crashes during WebAPI tests bug 859439 - will be easier to handle once crash detection is working here
  • [jgriffin] Make mach runners for B2G mochitests and reftests
  • [ahal] Enable crash detection on mozilla-b2g18 for mochitest/reftest
  • [ahal] Add crash detection to Marionette tests
  • [ahal] Write a script to download the latest emulator
  • [mdas] Use weakref in marionette-elements (bug 736592)
  • [mdas] Identify what needs to happen to get Gaia UI tests running on b2g desktop builds
  • [yiming] Complete touch/mouse event code for action chains
  • [yiming] Convert code from synthetic gestures to action chains
  • [rwood] Develop bluetooth Gaia UI test to pair unagi with host machine
  • [rwood] Finish integrating long running tests with datazilla

Milestone 2 - 2 weeks (tentative start: April 22)

  • Get Gaia UI tests running on B2G desktop builds in TBPL (will need a mozharness script)
  • Improve Marionette's error reporting with specific, clear error messages and stack traces
  • Work on other top B2G test usability problems and either fix or get them to a state where developers can take over

Milestone 3 - 2 weeks (planned)

  • Add debugger support to B2G testrunners

Milestone 4 - 2 weeks (planned)

TBD

Future milestones

TBD