Notes from the meeting the futurama team had with a few developers on 2012-10-26.

Problems from the Developers

• Number one is no neon support
  • [Futurama]: that is coming (panda boards)
• Windows 8
  • [Futurama]: releng (armen) working on that
• Having VMs would be nice - things that people could download so they could repro things
  • Download a VM with talos env on it so you can download it and reproduce the issue that talos hit
    • Have a VM with as close to a configuration to what's in production so we can run it similarly to the production environment. The goal isn't to reporoduce the same numbers, but to reporoduce the regression.
  • [Futurama]: identified as goal.
• Ability to run tests in parallel -- we want that for running locally.
  • reftests should be possible to parallelize they could be the next step after xpcshell (reftests that don't require focus)
  • jsreftests is another good candidate for this
  • perhaps a method of attack: xpcshell -> jsreftest -> crashtest -> reftest
  • [Futurama]: on buildbot, will probably cause a lot of oranges and suck up a lot of time, not a clear win. locally, bugs exist, could be a contributor project, but not a priority compared to other identified goals.
• More C++ tests.
  • XUL unit test
  • Integration of Gtest, helping make it easier to write tests for pieces that live inside of libXUL
  • Needs more work and investigation
  • The more information we can provide the better - we show backtraces when we crash that's useful.
  • reduces requirements for quick turn around time because you have enough data from one push and you don't need to push again to get more data.
  • But if we had profiling stats when we crash
  • It would be nice to have tests for some of this low level stuff - someone broke compilation error reporting on windows recently and that was difficult to understand and fix. This could be added to the "fast test suite" idea
  • [Futurama]: Identified as a (longer-term) goal.
• Improving quality of results
  • i.e. adding profiling with talos
    • [Futurama]: in progress.

Workflow issues

• Autoland is barely running - it's not something that is being worked on right now.
  • Need some method for checking in someting "whenever" - perhaps autoland can do that? For patches that are not time sensitive
  • Once the patch is r+ the patch should just land and the test results should show up in bugzilla automatically (the chrome/webkit automation story)
  • tbpl results on bug pages
  • [Futurama]: identified as goal.
  • Orange factor needs an owner to drive the number down
    • [Futurama]: edmorley working on that.
• Code coverage -- taking that further.
  • jcranmer has a setup that works on try. It's still a bit hackish but it is working, so that's a beginning. With more polish might be useful. Should be something to investigate with it.
  • http://people.mozilla.org/~choller/firefox/coverage/
  • [Futurama]: would be nice, but probably has a low ROI. Generates a lot of data but nothing clearly actionable. To make it useful, it would require a lot of tools and infrastructure. Long-term project that would require a lot of planning. Some goals we *have* identified may advance this on their own.
• ASAN builds - decoder has a way to run ASAN builds, it would be good to get nightly builds with it. So that people can download them and use them. Mostly to run locally.
  • [Futurama]: Building them is already possible. This would be a releng project; not much for A-Team to do here.

Bugzilla

• showed bugzilla dashboards - you can see them on https://bugzilla.allizom.org
  • [Futurama]: identified as a goal.
• The new skin is in the Preferences called "mozilla" skin drop down list at https://bugzilla-stage-tip.mozilla.org
  • [Futurama]: identified as a goal.
• Making bugzilla faster
  • There are changes to the backend to help with this:
    • Better backend hardware
    • Fixing search table inefficiencies
    • try to reduce the unnecessary load on bugzilla from the REST API
  • [Futurama]: we're chipping away at this.

Workflow Suggestions

• Ability to work in git locally and have a gitserver system that takes your pushes and then pushes into hg.
  • Someone has code for this -- follow up with Ehsan.
  • [Futurama]: Not really A-Team's purview, but in progress supposedly.

Need better docs/blogs about where things are

• what is mozbase? is it in mozcentral? where is the code?
• how do I do it?
• we need more links to the code from the running applications- make it very obvious.
• (SEO!) ensure our stuff shows up on google
• We should get rid of out of date crap - i.e. standalone talos.
• We should keep the projects page up to date and make it extremely obvious.
  • Just delete old/out of date information. If people really want to see history the wiki has a feature to let you view it.
  • [Futurama]: most of this was done shortly after the meeting! \o/