Marketplace/Testing: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
(added banner) |
||
| (7 intermediate revisions by 4 users not shown) | |||
| Line 1: | Line 1: | ||
{{Marketplace_banner}} | |||
== Overview == | == Overview == | ||
This is an overview of testing improvements for the Marketplace apps itself (zamboni, fireplace etc). | This is an overview of testing improvements for the Marketplace apps itself (zamboni, fireplace etc). | ||
=== Q1 Goals === | === Q1 Goals (completed) === | ||
* Improve testing of marketplace front end components (eg. marketplace-core-modules, marketplace-elements) | * Improve testing of marketplace front end components (eg. marketplace-core-modules, marketplace-elements) | ||
* Investigate using SlimerJS instead of Phantom | * Investigate using SlimerJS instead of Phantom | ||
==== Frontend Testing Changes ==== | |||
Unit tests will use [http://mochajs.org/ mocha] as the testing framework, [http://chaijs.com/ chai] for assertions, [http://sinonjs.org/ sinon] for mocks and stubs and run in the browser through [http://karma-runner.github.io/ karma]. These are already in use in spartacus and fxpay. | |||
=== Later goals === | === Later goals === | ||
=== | Things that might be a good idea: | ||
==== Visual regression testing ==== | |||
[http://github.com/ngokevin/sherlocked Sherlocked], a visual regression testing service, will be developed and integrated. | |||
* Attach images with large changes to PRs. | |||
==== Spin up a docker instance with changes for a PR ==== | |||
* When you submit a PR, spin up an AWS and load marketplace-env at your SHA. | |||
** This will aid manual testing (label:keep-test-env). | |||
** This will allow the integration tests to run against your PR. | |||
** Should this be app-specific? What if zamboni updates while your PR is open? | |||
* Video/screenshot of failing UI tests on Travis | |||
==== Testing visualization/metrics ==== | ==== Testing visualization/metrics ==== | ||
* Testing dashboard | * Testing dashboard (what metrics should we record?). | ||
* Comment on PR with coverage? (Likely with coveralls.io) | |||
* Comment on PR with coverage? ( | * Style testing bot similar GitHub's [http://zachholman.com/posts/how-github-writes-blog-posts/#continuous-integration blog tests]. These tests only run on changed code. | ||
* Style testing bot | |||
==== Integration tests across projects ==== | ==== Integration tests across projects ==== | ||
* Submission, review and consumer pages. | * Submission, review and consumer pages. | ||
** This exists @ https://github.com/mozilla/marketplace-tests/blob/master/tests/desktop/developer_hub/test_developer_hub_submit_apps.py | |||
* Purchasing an app. | * Purchasing an app. | ||
** This exists @ https://github.com/mozilla/marketplace-tests-gaia/tree/master/marketplacetests/payment | |||
* Likely casperjs but there is already a python repo from QA. | * Likely casperjs but there is already a python repo from QA. | ||
=== Bugs === | === Bugs === | ||
Latest revision as of 02:52, 1 April 2016
.png){kind=small}
The Marketplace has been placed into maintenance mode. It is no longer under active development. You can read complete details here.
Overview
This is an overview of testing improvements for the Marketplace apps itself (zamboni, fireplace etc).
Q1 Goals (completed)
- Improve testing of marketplace front end components (eg. marketplace-core-modules, marketplace-elements)
- Investigate using SlimerJS instead of Phantom
Frontend Testing Changes
Unit tests will use mocha as the testing framework, chai for assertions, sinon for mocks and stubs and run in the browser through karma. These are already in use in spartacus and fxpay.
Later goals
Things that might be a good idea:
Visual regression testing
Sherlocked, a visual regression testing service, will be developed and integrated.
- Attach images with large changes to PRs.
Spin up a docker instance with changes for a PR
- When you submit a PR, spin up an AWS and load marketplace-env at your SHA.
- This will aid manual testing (label:keep-test-env).
- This will allow the integration tests to run against your PR.
- Should this be app-specific? What if zamboni updates while your PR is open?
- Video/screenshot of failing UI tests on Travis
Testing visualization/metrics
- Testing dashboard (what metrics should we record?).
- Comment on PR with coverage? (Likely with coveralls.io)
- Style testing bot similar GitHub's blog tests. These tests only run on changed code.
Integration tests across projects
- Submission, review and consumer pages.
- Purchasing an app.
- Likely casperjs but there is already a python repo from QA.
Bugs
Tracking bug: 1127432.
No results.
0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);