QA/Test Automation/2010-07-13/SharedAPI
From MozillaWiki
< QA | Test Automation | 2010-07-13
Overall project principles:
- Scalability
- Ease of development
- Attractiveness to crowdsourcing
- Maintenance
- Clarity of test code
- Single point of change
- Minimizing change dependencies
- Tools
- Scalability
- Ease of development
- Create coherent domain-specific language/constructs for testing
- Adopt an extremely consistent model, where it's obvious where to look for things
- Document thoroughly. Should be obvious if a service exists, if so where, if not how to structure it.
- Push for tool changes/enhancements, as necessary, to smooth out development flow.
- Attractiveness to crowdsourcing
- Make it easy, as above. People mostly do easy things.
- Make sure documentation is easily findable and searchable.
- Ease of development
- Maintenance
- Clarity of test code
- Strong overlap with ease of development
- DSL for easy comprehension (what was the test doing yesterday?)
- Thorough documentation
- Minimize non-linear logic in tests. Logic should be in shared modules, where appropriate.
- Single point of change
- Should be targeted in model above
- All UI lookups should be encapsulated into single "frame" (object model, lookup strings, etc.)
- Minimizing change dependencies
- Recognize that there are two classes of code: utility code (i.e. setup, teardown, travel) and test code (actual test)
- Utility code -- order of operations unimportant. Corresponds to a function in CS sense: need this done, only care about result
- Test code -- order of operations important. Is implementing a UI scenario, done in a particular way. Corresponds to macro, care about the "how".
- Suggestion: Maintain two separate classes of modules, one for utility only, one for common test flows. Why?
- Two different code review requirements. We care much more about changing order of operations in test code.
- Two different dependency profiles. Wide dependencies on utility code, only area-specific dependencies on test code
- Two different algorithm. Utility code should be fastest, most reliable way to achieve operation. Test code is what's defined in test spec.
- Tools
- As Ease of development, better tools equal better maintenance. Debugging is optimized for here.
- Shared API concern: don't adopt structures that are very difficult to debug.
- Clarity of test code