User:Ashughes/Bugdays/20100913/Report

Etherpad

http://etherpad.mozilla.com:9000/bugweek-post-mortem

Accomplishments

• 94 bugs triaged
  • Bugday: 61 bugs
  • Testday: 33 bugs
• 45 testcases in Litmus
  • New: 38
  • Revised: 7
• Contributors: 11

Pre-participation Survey Findings

There are a few interesting findings from the results of the pre-bugweek survey. I've captured the results, with pretty pie-charts, in this report. The following are some of the highlights:

• Majority of testers are not in the US, using laptops at home, heavy internet users, and attending their first event
• Majority of testers are either newcomers to the community or have been around for at least a year
• Primary motivation is to learn and to make Firefox better
• No communication channel stands out as a clear winner
• Wide range of professional backgrounds, from forestry and psychology to software engineers
• Wide range of educational backgrounds from grade 3 to software development degree holders

Post-participation Survey Findings

There are a few interesting findings from the results of the post-bugday survey as well.

• testers participated at least a couple of the days for a couple of hours
• expectations of reward were low
• expectations of work/learning was high
• testers valued immediate feedback, 1:1 time, ability to win prizes
• for the most part, people were very helpful
• testers were likely to re-attend
• testers undervalued their contributions (thought their work was not worth the amount of prizes)
• areas for improvement:
  • testers found using the spreadsheet to be too cumbersome and technical
  • testers had issues using Mibbit
  • instructions for testcase summary were confusing
  • some mozQA people could be a bit less abrasive:
    • I created a test case and I entered the steps in IRC to ask if it was OK. They told me to enter it in the spreadsheet instead so as to avoid spamming in IRC. I understood but I didn't like the comment.

What Went Well

• Well received by community:
  • Workshops
  • 1:1 time
  • Focused work on new features
  • Competing events on Friday did not divide community, made it more focused

What Didn't Go So Well

• Advertisement didn't go out early enough
• Lower than anticipated MozQA participation
• Overall participation low for Session Restore and Tabbed Browsing

Lessons Learned

• offer more opportunity to learn
• provide 1:1 time
• need higher internal participation
• advertise early and often
• need to improve tools
• provide constructive, real-time feedback