ReleaseEngineering/Projects/Android 4.0 Testing

From MozillaWiki
Jump to: navigation, search

WIP - Draft warning

What is the goal?

  • bug 725544 - tracking bug for Android 4.0 testing
  • To test on android 4.0
  • To enable us to test NEON support (a graphics subsystem these cards have)
  • To test a different chipset architecture than the tegras
    • Tegra is largely a dying platform in mobile space
  • To have a mobile automation platform that is more reliable and easier to flash than the tegras (less IT management cost)
  • To have a mobile platform that can live in a lights-out datacenter
  • To have a mobile platform that is faster than the tegras

What does mobile need/know?

What does the a-team need/know?

  • How do we do flashing of these devices so that you can flash with all required software from the get-go
  • Does it run through our test systems without dying?
  • How much faster is it than a tegra?

What does IT need/know?

  • know: we don't have room for a production quantity of these in haxxor, only in real data-centers which necessitates a rackmount enclosure with proper remote management.
    • Q: does this rackmount exist? would it allow to replace SD cards on panda boards?
  • need: blocked on the exact specs on the board(s) we're ordering/supporting
    • A: ctalbert is going to let us know what he has. At the time there was only one model.
  • need: blocked on at least two physical boards/power supplies for enclosure prototyping
    • A: once ctalbert answers we can determine what to buy
    • must determine: how many panda boards can we fit in a custom 4U chassis, and how do we connect them to relay boards for remote administration?
    • how is imaging handled, will be able to do it remotely like we have planned with the tegras?
    • Q: how different is tegra current management compared to SD card replacement for pandas?
  • need: a defined timeline/gantt chart that shows when we'll need each phase done by and what it's requirements are (e.g. one board to play with in haxxor, N boards to play with in a datacenter (where N needs to be defined)), and a full blown production setup).
    • A: armenzg can help with estimates once we have timing on how fast unit/talos tests run on pandas VS tegras
    • A: probably 15 of them to do staging will help to test that imaging has worked and that we can run them on the staging setup.
    • A: blassey mentioned that having something running in March/April to get results going would be very useful to catch regressions
  • need: a good ballpark estimation of how many boards we will be putting into production and whether that is phased over several months or needs to be done all at once.
    • A: armenzg can help get a ballpark # once we have a machine running tests so we can time it.
    • how much space does this require in the dc
    • how many buildbot masters does this require, and of what configuration
    • how many foopies does this require, and of what type/configuration (minis are harder, linux is better, none is best)?
      • A: bear is going to get us Linux foopy VMs.
    • will these also connect to the tegra dashboard or something similar? Does that scale? what are the spec requirements for a machine/vm to make this work if this is the plan?
      • TBD
  • need: do 9 month cycle OS builds mean SD card swapouts/reimages every 9 months? What's that process look like? tree closure, rolling upgrades?
    • A: I (armenzg) think that putting a subset on the side to verify it is good and then we can do rolling downtime.

What does releng need/know?

Point of contact: armenzg

  • a panda board
  • hook up the panda-board to the automation
  • know if the setup is the same as tegras (SUT agent, foopies)
    • clint: Exact same setup.
  • what suites are going to be run on it
    • clint: Everything we run on the tegras
  • imaging process
    • Ateam to define
  • ordering schedule
  • how many boards?
    • do we really know if the model we follow is scalable?? does the tegra model work?
    • ordering these many boards is simply ridiculous for manufacturers
  • do they have to be racked in haxxor?
    • clint: No. These are wired devices, no reason to use haxxor for this.

Roadmap/Timeline

  • ateam gets for releng a working board in the week of February 12th
  • releng to setup a master/foopy to make this setup work - ETA?? owner??

Action items

  • ctalbert: will build an Android 4.0 OS image and flash to panda board. Largely doing this to see what the steps are, ensure they are clear and repeatable (Brad already proved it's possible) and to see where we'd modify these steps in order to create an OS with our tools pre-installed.
  • ctalbert: once he has the panda board running, he is going to do a quick sanity test run on the pandaboard to verify that it makes it through reftest/mochitest etc.
  • ctalbert: Once it runs through the tests without fizzling out, he is going to go rack it in haxxor so that the releng can put it into their staging environment and start doing preliminary tests on it to work through the changes that will be required in buildbot to handle another android device type reporting test results, as well as doing a true stability assessment.
  • data sheet - specs of panda board ES
  •  ??: provide requirements for dev, testing, and production phases to relops

Bugs/Links

Retrieved from "https://wiki.mozilla.org/index.php?title=ReleaseEngineering/Projects/Android_4.0_Testing&oldid=453588"