Mobile/Testing/03 21 12: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Created page with "= Previous Action Items = * Joel to take care of Ts-moz-afterpaint * finkle to give joel a list of perf tests he wants to write * ctalbert to flash new ES board with build and se...")
 
(Replaced content with "= Previous Action Items = = Status reports = == Dev team == == Rel Eng == == A Team == == S1/S2 Automation == == Mobile Crash == = Round Table = = Action Items =")
Line 1: Line 1:
= Previous Action Items =
= Previous Action Items =
* Joel to take care of Ts-moz-afterpaint
* finkle to give joel a list of perf tests he wants to write
* ctalbert to flash new ES board with build and see what new problems we find and what ends up fixed


= Status reports =
= Status reports =
== Dev team ==
== Dev team ==
* {{bug|735461}}: Robocop not using -profile argument consistently
* {{bug|734456}}: Reftest/Crashtest suites C1, R2, R3 hidden
** Overlap with {{bug|732498}}, etc.?
* Make it easy to use Robocop to add some UI talos performance tests for various UI pieces:
** Time to open the Awesome Screen
** Time to load basic Awesome Screen results
** Time to open Tabs List with X tabs open
** Time to display core UI
** Time to display about:home
== Rel Eng ==
== Rel Eng ==
* {{bug|723667}} - talos robocop
** sut agent upgrade patches being reviewed and soon to be landed
*** armenzg verifying with jmaher that sut agent upgrade does not cause regressions to other suites
** talos robocop jobs are still going red
*** jmaher got it working manually when its first run went red
*** armenzg has triggered another 3 jobs of each to see if anything changes


== A Team ==
== A Team ==
Robocop:
* not copying fresh profile over each test saves us about 1.5 minutes per test, or almost 20 minutes for a full run
** patch in progress, eta friday
* when launching via sutagent, we seem to take 30 seconds longer to run a test vs launching with adb.  A future area for investigation to save time
Reftests:
* in looking at the reftests that require larger resolution (I believe there are 10 of them), we do not run them on android and should consider not adjusting our resolution.  This will save us 10 minutes per reftest job or 180 minutes of machine time per checkin.
** side benefit is we will have more memory for fennec and might reduce our hangs or random failures (not proven or tested much)
* {{bug|732498}} - need somebody from development to help drive/fix the reftest failures between xul/native/e10s and android in general
Talos:
* {{bug|732206}} - maple ts and txul are busted, ts appears to be green the last few runs
* new talos.zip is being used, now we can iterate faster on talos changes
* need to do ts-mozafterpaint
Upcoming work:
* xpcshell (need new sutagent rolled out- in progress)
* powermeter
== S1/S2 Automation ==
== S1/S2 Automation ==
* public facing graphs will be available by end of the week


== Eideticker ==
== Mobile Crash ==
* automated tests are running
= Round Table =


== Crash Stats ==
== Panda Boards ==
* from ctalbert: "panda still needs a new build, going to rebuild it today"
=== Carry over info ===
concerns
* IT is concerned about not having remote re-imaging
** jake has ideas to try once he has time to use pxe
** this is not a high priority right now
* the boards sometimes become unstable (much more than the tegras)
** we have to remember that we are using tip of ICS and will take time to settle
* armenzg: do we have an alternative to the panda boards?
** blassey: no fallback plan
* {{bug|725544}} - Android 4.0 tracking bug
* [https://wiki.mozilla.org/Auto-tools/Projects/Pandaboard_Setup Pandaboard Setup docs]
== QA Robocop Coverage ==
* https://docs.google.com/spreadsheet/ccc?key=0AhE7m4JB2j6tdDJBT2dlbVJwUk9PSy1RbHo4WVNiUGc#gid=2
** Starter sheet
== Crash Stats ==
[[File:CrashStats_2012-03-12.png]]
* Screenshot and data from 3/12/2012 11:11 PM PDT
Top crashes:
* see [https://wiki.mozilla.org/Platform/2012-03-11#Mobile_2 Stability Report 2012-03-11]
Note :
* Socorro/Breakpad work:
** {{bug|726385}} Please create skiplist implementation for the Java field
** {{bug|726693}} Some java crashes are not being mapped correctly to bugs in bugzilla even though the crash signature fields have the "signatures" listed
** {{bug|672606}} - Aggregate numbers and topcrashes for Nightly and Aurora channels based on build ID date instead of crash date
** {{Bug|727234}} - Crash Report [@ EMPTY: no crashing thread identified; corrupt dump ]
** {{Bug|727286}} - Incorrect messaging when failing to submit a crash report
* ADU went up on March 7th a little bit due to missing pings; this was resolved on the 8th.
* 9th had changes to the libdvm so that they should now be reporting proper Java crashes.
= Round Table =
* We have to improve our purple/red VS orange/green ratio
** These bugs have been filed for long and are at the top of WOO
** {{Bug|724738}} - Intermittent Android mochitest and mochitest-robocop timeout
** {{Bug|686084}} - Intermittent Tegra "Cleanup Device failed" from "command timed out: 1200 seconds without output, process killed by signal 9"
** {{Bug|687116}} - Intermittent "unrecognized output format" running Talos on Android
** {{Bug|722166}} - Android tests intermittently just don't start


= Action Items =
= Action Items =
* [carry-over] Joel to take care of Ts-moz-afterpaint
* [carry-over] finkle to give joel a list of perf tests he wants to write
* [carry-over] ctalbert to flash new ES board with build and see what new problems we find and what ends up fixed
* joduinn to figure out plan for mobile mozharness moving forward since aki and armenzg are with full hands

Revision as of 17:10, 19 March 2012

Previous Action Items

Status reports

Dev team

Rel Eng

A Team

S1/S2 Automation

Mobile Crash

Round Table

Action Items

Retrieved from "https://wiki.mozilla.org/index.php?title=Mobile/Testing/03_21_12&oldid=408992"