Mobile/Testing/01 02 13: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Created page with "= Previous Action Items = = Status reports = == Dev team == == Rel Eng == == IT == == A Team == == Autophone == == Eideticker == = Round Table = = Action Items =")
 
m (→‎Autophone: removing stale brasstacks links)
 
(20 intermediate revisions by 8 users not shown)
Line 1: Line 1:
= Previous Action Items =
= Previous Action Items =
* disable reftest 1-4 for pandas on m-c
* allow panda jobs to be run on try server
* bring 198 pandas online for running on try and inbound


= Status reports =
= Status reports =
== Dev team ==
== Dev team ==
* Working on:
** {{bug|810471}} Crash on startup during various tests
** {{bug|811411}} Add ability to run C++ unit tests on mobile
** {{bug|811793}} Get at least a minimal subset of Android XPCShell tests running on tbpl
** Various robocop failures: bugs 770483, 813107, 817440, 824067
== Rel Eng ==
== Rel Eng ==
===Pandas===
* writing patches to enable 198 new pandas https://bugzilla.mozilla.org/824766 that were made available today


== IT ==
== IT ==
* Tracker bug - 800 production ready panda boards {{bug|799922}}
** All chassis are racked, issues with stability of the Android image when remotely flashed has been fixed. {{bug|819545}}
* Working on a higher density chassis {{bug|777393}} - This work has restarted.
== A Team ==
== A Team ==
* total failure rate
** android 2.2 [[https://docs.google.com/spreadsheet/ccc?key=0ArS97F99-BEZdHA5Y0NNdTQ5aTlSU0hNRWFJMWEzR1E 10.53%]]
*** most of the reds on rck, rck2, rpr, rp are related to nightlies failing which is fixed as of today
** android 4.0 [[https://docs.google.com/spreadsheet/ccc?key=0ArS97F99-BEZdE4yYXloeTJRSk9mdDhVQ0s5SFF5SkE 17.83%]]
*** slow progress, not many data points as this includes stuff from dec 22nd
* validated new lifeguard based installation method for android
* updating talos to have mozcrash and updated android/breakpad binaries
* plenty of robocop fixes/new tests
* landed patch to speed up robocop tests by 10+ minutes/job
== x86 automation ==
== Autophone ==
== Autophone ==
* Testrun on autolog: REMOVED


== Eideticker ==
== Eideticker ==
* New shinier faster dashboard code (with bonus b2g support) almost finished. Should be live in the next day or so.
* Work for getting notifications for when dashboard is giving wrong results still pending: https://bugzilla.mozilla.org/show_bug.cgi?id=823112. Hope to get to it this week.
* Some dashboard results were off because of a telemetry prompt change which interfered with the tests. Will probably just delete results for those days.
= Round Table =
= Round Table =
 
* fixing crashes
** do we need to report to socorro, desktop unittest crashes do not report there
** these account for the majority of the [[http://brasstacks.mozilla.com/orangefactor/index.html war on orange]] failures
** most are startup (usually java) and shutdown (*nss* related)
** tbpl->bugs are not helpful as it creates noise for crashes; what is idea, what can we start working on?
** {{bug|824760}} - generating core dumps?


= Action Items =
= Action Items =
* [callek] {{bug|824764}} - turn off reftest for panda
* [releng] {{bug|825984}} - turn on panda for try/inbound
* [kmoir] turn on 198 panda boards
* [jmaher] add x86 automation status to meeting notes
* [wlach] file a bug and investigate "push-to-eideticker"
* [jmaher] talk to socorro folks about reporting crashes from automation
* [jmaher] look into core dumps from android

Latest revision as of 15:08, 30 May 2017

Previous Action Items

  • disable reftest 1-4 for pandas on m-c
  • allow panda jobs to be run on try server
  • bring 198 pandas online for running on try and inbound

Status reports

Dev team

  • Working on:
    • bug 810471 Crash on startup during various tests
    • bug 811411 Add ability to run C++ unit tests on mobile
    • bug 811793 Get at least a minimal subset of Android XPCShell tests running on tbpl
    • Various robocop failures: bugs 770483, 813107, 817440, 824067

Rel Eng

Pandas

IT

  • Tracker bug - 800 production ready panda boards bug 799922
    • All chassis are racked, issues with stability of the Android image when remotely flashed has been fixed. bug 819545
  • Working on a higher density chassis bug 777393 - This work has restarted.

A Team

  • total failure rate
    • android 2.2 [10.53%]
      • most of the reds on rck, rck2, rpr, rp are related to nightlies failing which is fixed as of today
    • android 4.0 [17.83%]
      • slow progress, not many data points as this includes stuff from dec 22nd
  • validated new lifeguard based installation method for android
  • updating talos to have mozcrash and updated android/breakpad binaries
  • plenty of robocop fixes/new tests
  • landed patch to speed up robocop tests by 10+ minutes/job

x86 automation

Autophone

  • Testrun on autolog: REMOVED

Eideticker

  • New shinier faster dashboard code (with bonus b2g support) almost finished. Should be live in the next day or so.
  • Work for getting notifications for when dashboard is giving wrong results still pending: https://bugzilla.mozilla.org/show_bug.cgi?id=823112. Hope to get to it this week.
  • Some dashboard results were off because of a telemetry prompt change which interfered with the tests. Will probably just delete results for those days.

Round Table

  • fixing crashes
    • do we need to report to socorro, desktop unittest crashes do not report there
    • these account for the majority of the [war on orange] failures
    • most are startup (usually java) and shutdown (*nss* related)
    • tbpl->bugs are not helpful as it creates noise for crashes; what is idea, what can we start working on?
    • bug 824760 - generating core dumps?

Action Items

  • [callek] bug 824764 - turn off reftest for panda
  • [releng] bug 825984 - turn on panda for try/inbound
  • [kmoir] turn on 198 panda boards
  • [jmaher] add x86 automation status to meeting notes
  • [wlach] file a bug and investigate "push-to-eideticker"
  • [jmaher] talk to socorro folks about reporting crashes from automation
  • [jmaher] look into core dumps from android
Retrieved from "https://wiki.mozilla.org/index.php?title=Mobile/Testing/01_02_13&oldid=1172184"