Confirmed users
746
edits
Marketplace/Testing: Difference between revisions
added banner
(Created page with "== Overview == This is an overview of testing improvements for the Marketplace apps itself (zamboni, fireplace etc). === Bugs === Tracking bug: {{Bugzilla|1127432}}. <bugz...") |
(added banner) |
||
| (9 intermediate revisions by 5 users not shown) | |||
| Line 1: | Line 1: | ||
{{Marketplace_banner}} | |||
== Overview == | == Overview == | ||
This is an overview of testing improvements for the Marketplace apps itself (zamboni, fireplace etc). | This is an overview of testing improvements for the Marketplace apps itself (zamboni, fireplace etc). | ||
=== Q1 Goals (completed) === | |||
* Improve testing of marketplace front end components (eg. marketplace-core-modules, marketplace-elements) | |||
* Investigate using SlimerJS instead of Phantom | |||
==== Frontend Testing Changes ==== | |||
Unit tests will use [http://mochajs.org/ mocha] as the testing framework, [http://chaijs.com/ chai] for assertions, [http://sinonjs.org/ sinon] for mocks and stubs and run in the browser through [http://karma-runner.github.io/ karma]. These are already in use in spartacus and fxpay. | |||
=== Later goals === | |||
Things that might be a good idea: | |||
==== Visual regression testing ==== | |||
[http://github.com/ngokevin/sherlocked Sherlocked], a visual regression testing service, will be developed and integrated. | |||
* Attach images with large changes to PRs. | |||
==== Spin up a docker instance with changes for a PR ==== | |||
* When you submit a PR, spin up an AWS and load marketplace-env at your SHA. | |||
** This will aid manual testing (label:keep-test-env). | |||
** This will allow the integration tests to run against your PR. | |||
** Should this be app-specific? What if zamboni updates while your PR is open? | |||
* Video/screenshot of failing UI tests on Travis | |||
==== Testing visualization/metrics ==== | |||
* Testing dashboard (what metrics should we record?). | |||
* Comment on PR with coverage? (Likely with coveralls.io) | |||
* Style testing bot similar GitHub's [http://zachholman.com/posts/how-github-writes-blog-posts/#continuous-integration blog tests]. These tests only run on changed code. | |||
==== Integration tests across projects ==== | |||
* Submission, review and consumer pages. | |||
** This exists @ https://github.com/mozilla/marketplace-tests/blob/master/tests/desktop/developer_hub/test_developer_hub_submit_apps.py | |||
* Purchasing an app. | |||
** This exists @ https://github.com/mozilla/marketplace-tests-gaia/tree/master/marketplacetests/payment | |||
* Likely casperjs but there is already a python repo from QA. | |||
=== Bugs === | === Bugs === | ||