Mobile/Testing/06 25 14

From MozillaWiki
Jump to: navigation, search

Previous Action Items

  •  :snorp to send :bc info for his custom autophone needs
  • someone(!) should file bugs to capture ideas for additional autophone features discussed today: make it easier to run locally, autophone via try, etc.

Status reports

Dev team

Rel Eng

  • We have a lot of dead Tegras (~100). Callek is wondering if it is worth doing another sutagent deploy given their EOL. The problems they are exhibiting is that they aren't allowing a connection on the SUTAgent ports.
  • reconfig to enable 2.3 tests on armv6 and ride the trains failed last night despite passing in staging and pre-production tests. Investigated and it requires a rolling restart of certain test masters to implement in production.
  • we may start using a more powerful AWS instance type (c3.large) for selected tests. This will allow some b2g tests currently running on ix move to AWS. Not sure if this will help us move more 2.3 tests to AWS.

Relops

A Team

  • android2.2 Total failure exclude retry rate: [3.26%]
    • last week 7.66%
  • android2.3 Total failure exclude retry rate: [0.22%]
    • last week 0.25%
  • android4.0 Total failure exclude retry rate: [17.06%]
    • last week 15.19
    • total failures excluding retries 2.33%

Android 2.3 emulator

  • Needed to disable more Robocop tests for aws. Tests disabled on 2.3:
       testAboutHomeVisibility (aws) 946656
       testAboutPage bug 975187 -> bug 1025968
       testAddSearchEngine bug 979552
       testBookmarkFolder bug 979552
       testBookmarksPanel bug 979615
       testClearPrivateData bug 948591
       testDoorHanger (aws) bug 1028929
       testImportFromAndroid bug 979552 -> bug 1025968
       testInputConnection bug 1025968
       testMasterPassword bug 979603
       testNewTab (aws) bug 995696
       testOverscroll (aws) bug 836818
       testSelectionHandler bug 980074 -> bug 1025968
       testSessionOOMRestore (aws) bug 979600
       testSessionOOMSave (aws) bug 945395
       testSettingsMenuItems bug 979552 -> bug 1025968
       testSystemPages bug 979603 -> bug 1025968
       testTitleBar bug 979552 
  • A few random mochitest timeouts noted on aws -- watching frequency.
  • media mochitests?

Autophone

  • Are we ready to call Autophone and Eideticker our official perf frameworks?
    • [bc] Now that the wifi situation in the QA lab has settled down, I need to investigate getting the remote tests running again.
  • Ran tests for 2014-06-21 through part of the day 2014-06-22 with and without ondemand linking and forwarded the results to snorp. See ondemand testing.
  • Retested webappstartup.
    • snorp noticed a marked decrease in times on 2014-06-20. I had deployed an update to Autophone on that day which may have been responsible for the change. I retested the time range with the production devices posting to phonedash-dev and found the previous results prior to the 20th were too high in general. This may have been due to an error during the initial deployment of the webappstart up. I cleared the results and have submitted jobs to backfill to the 18th. The decrease in start and stop time is still apparent for the nexus s devices, but no longer for the remainder of devices.
  • nexus-5-kot49h-2 and nexus-7-jdq39-1 both went offline probably due to battery drain due to the load caused by the backfilling. Will have back up later today.
  • Waiting final reviews for bug 1012711 for adb.py for mozbase and finalizing patch for bug 1007220 for the final webappstartup deployment.

Eideticker

  • We may actually set up the Galaxy Nexus in Mountain View after all; we got HDMI working with it and the network seems to be better than before

Round Table

Action Items

  •  :kmoir will ping blassey when armv6 reconfig done