Mobile/Testing/04 03 13: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Created page with "= Previous Action Items = blassey - PGO (gbrown), feedback on 3G emulation (mcote) wlach - Report back on regression = Status reports = == Dev team == == Rel Eng == == IT ==...")
 
 
(14 intermediate revisions by 7 users not shown)
Line 6: Line 6:
== Dev team ==
== Dev team ==
== Rel Eng ==
== Rel Eng ==
* Callek: xpcshell tests are live on cedar and are easy to turn on for other trees (try, inbound/central).  Have a high setup cost (30 minutes) but our test capacity can handle this currently {{bug|828069}}
* kmoir:  Landed change to run mochitests slower which should diminish the number of panda reboots until power supply issues are addressed {{bug|811444}}


== IT ==
== IT ==
* Still working on a higher density chassis, parts coming in to build a prototype.
* Issues with pandas rebooting may be lack of sufficient power.  3 new power supplies installed to test this theory.  3 weeks lead time from the manufacturer to get more
== A Team ==
== A Team ==
* failure rates
** tegra: [[https://docs.google.com/spreadsheet/ccc?key=0ArS97F99-BEZdG9rSjltVnhPWFp1bTZ2SVZyR2NELUE&usp=sharing 4.59%]]
*** mochitest 1,2 and robocop are the biggest offenders
** panda-all: [[https://docs.google.com/spreadsheet/ccc?key=0ArS97F99-BEZdGtNOTRhcC1uTjdBSUFIa1JzeG9rVHc&usp=sharing 18.52%]] - this is raw including robocop which is hidden on tbpl
** panda: [[https://docs.google.com/spreadsheet/ccc?key=0ArS97F99-BEZdGtNOTRhcC1uTjdBSUFIa1JzeG9rVHc&usp=sharing 4.35%]]
*** jsreftest 1,2 and mochitest 7 are the biggest offenders
* panda board reboot problems
** hypothesis is that our power supplies to not provide enough power, adding a second power supply will reduce our reboot and connectivity problems
** tested in chassis #2 before and after power supply addition, saw great improvements (12% to 2% failure rate)
** been working on testing in new chassis. 
*** Before I was seeing 12-25% failure rate on the smoketests
*** now I am seeing 4.6%, 7.5%, 9.0% running the same tests.
== x86 automation ==
== x86 automation ==
* Possibly obtaining x86 phone for Autophone testing.


== Autophone ==
== Autophone ==
* setup bandwidth limiting per device via:
<pre>sudo ipfw pipe $pipe config bw 780Kbit/s delay 100 plr 0
sudo ipfw add $pipe pipe $pipe ip from $hostip to $ip out
let pipe=pipe+1
sudo ipfw pipe $pipe config bw 330Kbit/s delay 100 plr 0
sudo ipfw add $pipe pipe $pipe ip from $ip to $hostip in
</pre>
* reran s1s2 tests for 2012-10-01 to date for 8 phones. Results available at: http://mrcote.info/phonedash/


== Eideticker ==
== Eideticker ==
* Still trying to get a finer regression range for CNN page load on LG G2X ({{bug|855019}}).
* Finally got cross browser page load test up. Found that our performance was bad, filed {{bug|857359}} to track.
* Discussed with jchen idea of a new checkerboarding metric using frequency content analysis of capture frames for sharpness/density. Filed {{bug|857644}} to cover investigation.
= Round Table =
= Round Table =
 
* debug tests
** need to resolve tooling https://bugzilla.mozilla.org/show_bug.cgi?id=818576#c28
* xpcshell tests


= Action Items =
= Action Items =
* wlach - Report back on regression
* jmaher - test new power supplies

Latest revision as of 17:36, 3 April 2013

Previous Action Items

blassey - PGO (gbrown), feedback on 3G emulation (mcote) wlach - Report back on regression

Status reports

Dev team

Rel Eng

  • Callek: xpcshell tests are live on cedar and are easy to turn on for other trees (try, inbound/central). Have a high setup cost (30 minutes) but our test capacity can handle this currently bug 828069
  • kmoir: Landed change to run mochitests slower which should diminish the number of panda reboots until power supply issues are addressed bug 811444

IT

  • Still working on a higher density chassis, parts coming in to build a prototype.
  • Issues with pandas rebooting may be lack of sufficient power. 3 new power supplies installed to test this theory. 3 weeks lead time from the manufacturer to get more

A Team

  • failure rates
    • tegra: [4.59%]
      • mochitest 1,2 and robocop are the biggest offenders
    • panda-all: [18.52%] - this is raw including robocop which is hidden on tbpl
    • panda: [4.35%]
      • jsreftest 1,2 and mochitest 7 are the biggest offenders
  • panda board reboot problems
    • hypothesis is that our power supplies to not provide enough power, adding a second power supply will reduce our reboot and connectivity problems
    • tested in chassis #2 before and after power supply addition, saw great improvements (12% to 2% failure rate)
    • been working on testing in new chassis.
      • Before I was seeing 12-25% failure rate on the smoketests
      • now I am seeing 4.6%, 7.5%, 9.0% running the same tests.

x86 automation

  • Possibly obtaining x86 phone for Autophone testing.

Autophone

  • setup bandwidth limiting per device via:
sudo ipfw pipe $pipe config bw 780Kbit/s delay 100 plr 0
sudo ipfw add $pipe pipe $pipe ip from $hostip to $ip out
let pipe=pipe+1
sudo ipfw pipe $pipe config bw 330Kbit/s delay 100 plr 0
sudo ipfw add $pipe pipe $pipe ip from $ip to $hostip in

Eideticker

  • Still trying to get a finer regression range for CNN page load on LG G2X (bug 855019).
  • Finally got cross browser page load test up. Found that our performance was bad, filed bug 857359 to track.
  • Discussed with jchen idea of a new checkerboarding metric using frequency content analysis of capture frames for sharpness/density. Filed bug 857644 to cover investigation.

Round Table

Action Items

  • wlach - Report back on regression
  • jmaher - test new power supplies
Retrieved from "https://wiki.mozilla.org/index.php?title=Mobile/Testing/04_03_13&oldid=643721"