Summary

This page tracks planning for the development of a community around the Mozilla Iris test automation project.

At present time, and since the creation of this wiki page, the following hurdles are known to exist within the project:

• biggest hurdle is structure and workflow due to refactor work
• logistical issue to on-board a new person
• would be nice to have a goal and understanding of how you're going to measure it (eg. X number of people involved) and what impact we expect that will have / why we need it
• need to think about impact of losing paid developers
• people who stumble in will contribute in some minor ways and not a result of something we have deliberately done
• we need to identify customers (mobile and other projects at Mozilla), invite you to a personal hack session to show you how to build a module and tests for your project. Rapid visual automation for your product.
• Another idea is to get someone who tests products in an emulator (Focus, Fennec, etc) and get them to hack with us on it.
• target people in the QA Org from phonebook - ask what they use for testing, how we could get them using Iris
• for second half of the year, if you can travel in North America, could use somebody who could take a trip to a college, youth group, hacker space, conference, etc. Give a hands on workshop with a local group.
• barrier to entry seems to be low, someone found us
• need an entry survey
• need to understand how people are discovering us
• identify issues and nice to haves through conversations with customers, use these issues to direct community
• blanket advertising won't work very well, needs to be focused on a specific area

The goal is to foster a healthy community of developers and customers alike.

Possible Metrics

Activity provides a first view of how much the community is doing, and can be used to track different kinds of activity.

• number of commits gives a first idea about the volume of the development effort.
• number of tickets opened provides insight into how many bugs are reported or new features are proposed.
• number of messages in mailing lists or posts in forums gives an idea of how much discussion is being held in public.

Size of the community is the number of people participating in it, but, depending on the kind of participation, size numbers may vary.

• number of active contributors (any and all people contributing code or otherwise to the project)
• number of core contributors (fraction of people contributing large proportions of code to the project)
• number of lead contributors (fraction of people contributing as leaders within the project)

Performance analyzes how processes and people are performing, and whether or not the project has sufficient resources.

• mean time to resolve or close tickets
• mean time spent in code review
• ratio of new to triaged to closed tickets

Demographics measures how people enter and leave a community over time, and the tenure of the community.

• number of new people joining during the corresponding period of time.
• number of active people still in the community broken down by "generations" based on tenure
• retention rate of contributors in each "generation" based on tenure

Diversity measures the resiliency of the community in terms of people and organizations participating.

• ratio of employee to volunteer contributors
• ratio of employee to volunteer contributions
• minimum number of developers performing 50% of the commits
• minimum number of employees performing 50% of the commits

Beyond the above, the Iris project should take a look at CHAOSS Metrics to evaluate which are relevant to the sustainable growth of the project. In addition the project needs to take a look at how it overlaps with other initiatives and communities within Mozilla so that we are supporting each other, not competing with each other; both in terms of users and contributors.

Phase 1: SWOT Analysis

For more background on SWOT Analysis as it pertains to Community Organization, see Wikipedia.

Strengths and weaknesses

Internal factors within an organization:

• Human resources — staff, volunteers, board members, target population
• Physical resources — your location, building, equipment
• Financial — grants, funding agencies, other sources of income
• Activities and processes — programs you run, systems you employ
• Past experiences — building blocks for learning and success, your reputation in the community

Opportunities and threats

External factors stemming from community or societal forces:

• Future trends in your field or the culture
• The economy — local, national, or international
• Funding sources — foundations, donors, legislatures
• Demographics — changes in the age, race, gender, culture of those you serve or in your area
• The physical environment —is your building in a growing part of town? Is the bus company cutting routes?
• Legislation — do new federal requirements make your job harder...or easier?
• Local, national, or international events

Elements to consider

Elements to consider in a SWOT analysis include understanding the community that a particular organization is working with. This can be done via public forums, listening campaigns, and informational interviews. Data collection will help inform the community members and workers when developing the SWOT analysis. A needs and assets assessment is tooling that can be used to identify the needs and existing resources of the community. When these assessments are done and data has been collected, an analysis of the community can be made that informs the SWOT analysis.

Steps for implementation

A SWOT analysis is best developed in a group setting such as a work or community meeting. A facilitator can conduct the meeting by first explaining what a SWOT analysis is as well as identifying the meaning of each term. One way of facilitating the development of a SWOT analysis includes developing an example SWOT with the larger group then separating each group into smaller teams to present to the larger group after set amount of time. This allows for individuals, who may be silenced in a larger group setting, to contribute. Once the allotted time is up, the facilitator may record all the factors of each group onto a large document such as a poster board, and then the large group, as a collective, can go work through each of the threats and weaknesses to explore options that may be used to combat negative forces with the strengths and opportunities present within the organization and community. A SWOT meeting allows participants to creatively brainstorm, identify obstacles, and possibly strategize solutions/way forward to these limitations.

Phase 2: ???

Phase 3: Profit

Other Considerations

• Select a communication platform (Slack, Discourse, Github, etc)
• Where possible, use bots for leaderboards, to welcome newcomers, to help people find tasks, and to enable people to engage
• Ensure you have a code of conduct, create a safe space for people to work how they want to work and to make mistakes
• Help your helpers: get them access to core Engineering resources, invite them to chats/calls to further their understanding and engagement, point them towards programs to collaborate with individuals. The more community members turn into experts, and the more experts turn into professionals on your platform, the better it is for everyone.
• Create contests to spur rapid development of a specific deliverable
• Blog about your project and community, encourage those in your community to do the same, and amplify those posts.
• Periodically feature a member of the community
• Send swag, stickers, etc of your community to your community
• Think about how you're helping to build careers
• Periodically reflect on the state of your community, identify areas for improvement and commit to solving one problem.
• Set Goals and Measure Results. How will you benefit from this collection of potential advisors, critics, contributors, and customers? What are your critical metrics, and how can you measure them? Identify your main goal: Is it software downloads? Getting developers to contribute to your project? Adding names to your developer mailing list? For example, consider using the “Unique clones” number included in the Traffic Statistics section of the GitHub repo reports as it'll give you a good idea of your daily download activity. Also log all of the GitHub repository statistics on a regular basis: Stars, Forks, Issues, Pull Requests as these statistics give you usage “signals.”
• Estimate future marketing activities and product evolution and put a straw-man plan in place, this will give you a way to measure progress against any plan.
• Consider the tech you're using and how you can utilize external communities centred around that tech to bolster your project (eg. Hackernews)
• Excellent documentation is the first step in making your software easy-to-use, make sure you have readmes, the API, and supporting technical documentation in place.
• Code of conduct: In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to make participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
• Make it easy for developers to ask and answer questions. (IRC is one of the primary channels we use. There is usually one of our developers “lurking” around there somewhere.)
• Identify issues for first-timers
• Documentation edits are a great way to gain experience in an open source project.
• Getting the opinions of your users reveals whether your code is meeting its big-picture goals.
• In addition to user interviews, attend conferences and meetups to build awareness and find the right audience. Even when you aren’t presenting, attending these events and speaking to developers directly is a great way to get feedback. Face-to-face interactions often help us build a better understanding of the problems and challenges that developers face.
• Listen to feedback