QA/Execution/Web Testing/Project for Automation: Difference between revisions
No edit summary |
|||
| Line 6: | Line 6: | ||
While the prospect of automation often sounds like it'd improve any project, unfortunately that is not possible to do. The bottom line is that there needs to be a vetting process as the team does not have the resources to automate every project. There are qualities which make a project a better candidate for automation. | While the prospect of automation often sounds like it'd improve any project, unfortunately that is not possible to do. The bottom line is that there needs to be a vetting process as the team does not have the resources to automate every project. There are qualities which make a project a better candidate for automation. | ||
Urgency: What is the urgency of your project? This includes whether it's a high priority for the company or if it has a short deadline. Both of these need to be considered. | <strong>Urgency: What is the urgency of your project? This includes whether it's a high priority for the company or if it has a short deadline. Both of these need to be considered. | ||
Lifespan: Is your project permanent? Is it going to be a maintained project that will last a year or more? Or is this a short term project? The lifespan of a project does impact the eligibility of it for automation. | Lifespan: Is your project permanent? Is it going to be a maintained project that will last a year or more? Or is this a short term project? The lifespan of a project does impact the eligibility of it for automation. | ||
Revision as of 19:22, 18 March 2013
How To: Getting a project Automated
Web QA has a large number of automated projects. Tests which are created and maintained by the Web QA team. This document will outline the process to get a project in the queue for Web QA automation.
Eligibility:
While the prospect of automation often sounds like it'd improve any project, unfortunately that is not possible to do. The bottom line is that there needs to be a vetting process as the team does not have the resources to automate every project. There are qualities which make a project a better candidate for automation.
Urgency: What is the urgency of your project? This includes whether it's a high priority for the company or if it has a short deadline. Both of these need to be considered.
Lifespan: Is your project permanent? Is it going to be a maintained project that will last a year or more? Or is this a short term project? The lifespan of a project does impact the eligibility of it for automation.
UI Stability: Is the project in a stable phase, or is it new to development? More stable projects are better candidates for automation. A changing UI makes for a higher intensity of development to keep up with the evolution of the site. Revamping a site's UI may be a good time to create or edit automation also.
Coverage Complexity: Are you requesting to have a short smoke test automated, or in-depth coverage of a full product? How much needs to be done, and how quickly? These are important elements for automation. Team resources need to be scoped out, as far in advance as is possible.
Discussion: The bottom line is that there cannot be any cut and dry formula to determine if a project is a good candidate for automation. There is a lot that depends on the project, its urgency, and the resources the team has available. Web QA wants to be available as a resource, however it's just not possible to always say 'Yes', which is why there is a need for discussion. Even if it is not possible to do right now, there is a possibility that eligibility may change in the future. For example we initially said no to marketing projects, however since then we have created a short template for those projects that actually carved off time for manual testing. So please talk to us! Tell us what you have and what you need, and we will do our best to accommodate you.
What next?
After your project has been nominated for automation, and resources have been scheduled these are the next steps:
- Project creation on Github: Mozilla Admins have this permission. If it is unclear who should be ask, request project submission from StephenD.
- Jenkins : Access is based on LDAP. Any team member can get access to view the Jenkins project dashboard. Currently Jenkins can be set to pull SCM (source code management), and given how many minutes for the interval. It is given the developer's GitHub repo and then goes to the dev site every 15-35 minutes to look for changes. Due to firewall issues, it is not possible at this time to do a pull when there is a change rather than a timed interval.
- Test creation: Project requirements, including test coverage requests will be discussed by the team. Resources will be based on availability and need.
- Test maintenance: Test maintenance can be done by QA or team members. This can be determined on a per-project basis.
- IRC notifications: IRC can be set up to notify the project team in any channel when the project status changes. For example, if it's been failing and suddenly passes it will generate a notification. Alternately it will notify the channel if tests which have been passing suddenly fail. You can check the status of any project by this IRC command, "qatestbot: status name.of.job [ex: qmo.prod]"