QA/Execution/Web Testing/Automation tools

List of Automation tools

Evaluations of current Automation tools, languages, tools for continuous delivery.

Big picture questions:

• When do you get a new framework or language?
• When is it time to move away from how things have been done?
• Risk of complexity and instability of going with new tools vs. Risk of irrelevancy and being left out of the process

Continuous Integration

• Jenkins
  • Open to all contributors
  • Considered to be outside of the project development release process

• Travis CI

Data collection tools

• Datadog
• Google Analytics

Site Monitoring tools

• New Relic

Languages

• JS
  • More support from developers who prefer the language
  • Async testing framework
  • Scripts vs. Page Objects

• Python
  • More support from Web QA
  • More options for testing using pytest

Load Testing

• Bees with Machine Guns
  • Forked improved version, rather than original
  • https://irccloud.mozilla.com/pastebin/RjGUbTDs/.boto

• Locust

Headless testing

• Are page renderings being tested?
• What approximation to a "real" browser do these frameworks represent?
• Are these proper
• Value of running speed

Casper.js

• Uses Phantom.js
• Opinion: Easy to learn and set up, simple API
  • Easy to use CSS selectors
• Not a good replacement for cross browser testing
• No page object model in tests, removes a level of abstraction & removes consistency found in page objects.
• Runs very fast in dev/stage
• Runs very slow locally as each page must load
• Can only check basic functionality; example: plugin check page
• Does not check redirects reliably

? Can it run on Travis?

Slimer.js

• runs using Casper

Webdriver/Selenium

Intern.io

• Opinion: Verbose API

Webdriver JS framework

• Opinion: better than Intern, not as complete as Selenium/Python