Marketplace/Testing

Overview

This is an overview of testing improvements for the Marketplace apps itself (zamboni, fireplace etc).

Q1 Goals

• Improve testing of marketplace front end components (eg. marketplace-core-modules, marketplace-elements)
• Investigate using SlimerJS instead of Phantom

Frontend Testing Changes

Unit tests will use mocha as the testing framework, chai for assertions, sinon for mocks and stubs and run in the browser through karma. These are already in use in spartacus and fxpay.

Later goals

Things that might be a good idea.

Testing visualization/metrics

• Testing dashboard (what metrics should we record?).
• Comment on PR with coverage? (Likely with coveralls.io)
• Style testing bot similar GitHub's blog tests. These tests only run on changed code.

Integration tests across projects

• Submission, review and consumer pages.
  • This exists @ https://github.com/mozilla/marketplace-tests/blob/master/tests/desktop/developer_hub/test_developer_hub_submit_apps.py
• Purchasing an app.
  • This exists @ https://github.com/mozilla/marketplace-tests-gaia/tree/master/marketplacetests/payment
• Likely casperjs but there is already a python repo from QA.

Visual regression testing

• Attach images with large changes to PRs.

Spin up a docker instance with changes for a PR

• When you submit a PR, spin up an AWS and load marketplace-env at your SHA.
• This will aid manual testing (label:keep-test-env).
• This will allow the integration tests to run against your PR.
• Should this be app specific? What if zamboni updates while your PR is open?

Video/screenshot of failing UI tests on Travis.

Bugs

Tracking bug: 1127432.

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);