QA/Test Automation/2010-07-13/SharedAPI

From MozillaWiki
Jump to: navigation, search

Shared API Overhaul

Overall project principles:

  1. Scalability
    1. Ease of development
    2. Attractiveness to crowdsourcing
  2. Maintenance
    1. Clarity of test code
    2. Single point of change
    3. Minimizing change dependencies
    4. Tools

How Shared API can help achieve:

  1. Scalability
    1. Ease of development
      1. Create coherent domain-specific language/constructs for testing
      2. Adopt an extremely consistent model, where it's obvious where to look for things
      3. Document thoroughly. Should be obvious if a service exists, if so where, if not how to structure it.
      4. Push for tool changes/enhancements, as necessary, to smooth out development flow.
    2. Attractiveness to crowdsourcing
      1. Make it easy, as above. People mostly do easy things.
      2. Make sure documentation is easily findable and searchable.
  2. Maintenance
    1. Clarity of test code
      1. Strong overlap with ease of development
      2. DSL for easy comprehension (what was the test doing yesterday?)
      3. Thorough documentation
      4. Minimize non-linear logic in tests. Logic should be in shared modules, where appropriate.
    2. Single point of change
      1. Should be targeted in model above
      2. All UI lookups should be encapsulated into single "frame" (object model, lookup strings, etc.)
    3. Minimizing change dependencies
      1. Recognize that there are two classes of code: utility code (i.e. setup, teardown, travel) and test code (actual test)
      2. Utility code -- order of operations unimportant. Corresponds to a function in CS sense: need this done, only care about result
      3. Test code -- order of operations important. Is implementing a UI scenario, done in a particular way. Corresponds to macro, care about the "how".
      4. Suggestion: Maintain two separate classes of modules, one for utility only, one for common test flows. Why?
        1. Two different code review requirements. We care much more about changing order of operations in test code.
        2. Two different dependency profiles. Wide dependencies on utility code, only area-specific dependencies on test code
        3. Two different algorithm. Utility code should be fastest, most reliable way to achieve operation. Test code is what's defined in test spec.
    4. Tools
      1. As Ease of development, better tools equal better maintenance. Debugging is optimized for here.
      2. Shared API concern: don't adopt structures that are very difficult to debug.