What NOT To Change

Technical Acumen

• Ability to focus on one thing so that you become an expert
• Keep using data to back up bugs.
• engaging with development on usage of tools and knowledge sharing
• dilligence in keeping smoketest automation alive
• Flashing tool that the taipei team made (need more like that)

Data Metrics

• identifying broad areas that people can insert themselves into/knowledge that people need to have to get involved.
• keeping track of crash stats (good focus from all teams)
• amount of metrics we collect from the users - telemetry, fx health crash etc.
• tracking and responding to user feedback pretty quickly

Relations - Community/Internal

• how we help each other
• keep clint
• keep all of us
• relationship with developer s(webqa, android, etc)
• having a number of testers on nightly Firefox browsers
• healthy dissatisfaction
• keep interest in community members
• continue engaging with other teams like relman on process/policy improvements
• Keep organizing test days

   * Get more creative with test days

   Good at explaining what QA does what value we bring and what we can make better

   Recognizing the community that contributes to QA

Process

• good analysis of what went wrong, post mortem.
• keep creating oppotunities - think of something someone can help out with.
• Keeping a bird's eye view on the project and understanding all its parts
• being a part of the development process - successful embedding with the teams (web)
• keep asking the hard questions
• keeping an eye on phones after they go to market
• conitnue representing the qa interest at events whatever shape they take
• surfacing concerns early -- issues with products
• Working more in the open by default
• Generating proposals for change (seeing what is wrong)
• Generating change
• The way we do face to face communication to clarify things more clearly, mitigate misunderstandings
• Have one tool that dumps all relevant information from a device

THINGS WE WANT TO CHANGE

Technical Acumen

• Knowledge about devices and OS's should be more readily available
• (1) TIME
  • Having time to explore new tools/try new ideas
  • Explicit time to breathe/make mistakes
• Break rule of build machines strictures
• More tools to support automation and developers
  • Evangelizing tools for devs so that they use and respect the tools that we have
• Automated tests by dev should be reviewed by QA & QA tests should be reviewed by dev
• More exploratory testing less manual tests over and over

Data/Metrics

• Use more of the user data we collect
• Way we provide/view crashdata is treated secondary to what our partners present
• Not enough metrics around b2g crash data/stability data
• Make metrics easier to digest/discover etc
• (1) Focus metrics around what question we want answered
• What is the right set of things to verify/we have time for/is useful to do/have infra to do so/what are you trying to accomplish
• Some test reports are sent by email, would be better to host in a database to do stats and history on them
• make reporting from moztrap easier

Relations - community/internal

• Picking the right battles.
• (1) Lower the bar to entry to include as many people as possible yet keep the tasks as rewarding as possible
• Knowing who to talk to
• Gracefully exiting meetings that are no longer useful
• Have test days in other languages than english
• + Need a PR campaign for QA -- too often devs don't know who we are
• + More storytelling - both internally and externally - brownbags/meetups/blogs/etc
• Good to have a way to share feedback or criticism that wasn't a meeting or an etherpad
• Improve cooperation between groups/subgroups - i.e. sharing dashboarding, or visualizatins based on jenkins data work
• Community manager for QA
• Listening to the Community

Internal

• (1) Knowing who to talk to
• PR for QA

Process

• (1) DOCS
• (2) PART OF DECISION
• + GOALS
• + CONTRACTORS - 4pm Thursday
• Proceed from goal to solution
• Not letting traditional processes from trying new ideas - the old ways aren't necessarily better, they are often just old.
• Set more aggressive bar for when our tools/processes aren't working right (ex moztrap issues & qmo)
• Ensure we have clearly defined set of objectives for all the goals that we set
• Make sure partners ship devices to have crash reporting turned on
• Get better about changing the rules - the game
• Better way to scale everything
  • Too dependent on partner IP issues and hung up on them (b2g)
• Out of date docs
• Have freedom from constant pressure/never going to catch up
• Never not in triage mode
• More QA activity at the beginning of the feature development
• Make significant changes that are measured against gains we want to make
• Open up as much infra as possible to everyone that can safely use it learn from it benefit from it
• The way that partners file bugs in bugzilla
• Contractors - have them too much like a crutch
• Tasks in bugzilla not diff'ing from bugs
• Stale uncentralized docs
  • Multiple doc accounts required
• Being part of the decision instead of complaining afterwards
• IRC questions go unanswered
• Answering volunteer quesitons especially outside of business hours
• Set boundaries around what we can deliver and ask other teams to prioritize their ask
• Getting more teams to follow the firefox front end dev team process
• Have developers write and own unit tests