Marketplace/Testing: Difference between revisions

no edit summary
No edit summary
No edit summary
Line 14: Line 14:
Things that might be a good idea.
Things that might be a good idea.


==== Testing visualization/metrics ===
==== Testing visualization/metrics ====
1. Testing dashboard.
* Testing dashboard.
2. Graphs showing # of tests.
* Graphs showing # of tests.
3. Comment on PR with coverage? (I don't like coverage)
* Comment on PR with coverage? (I don't like coverage)
4. Style testing bot a la GitHub? http://zachholman.com/posts/how-github-writes-blog-posts/#continuous-integration
* Style testing bot a la GitHub? http://zachholman.com/posts/how-github-writes-blog-posts/#continuous-integration
   
   
==== Integration tests across projects ====
==== Integration tests across projects ====
1. Submission, review and consumer pages.
* Submission, review and consumer pages.
2. Purchasing an app.
* Purchasing an app.
3. Likely casperjs but there is already a python repo from QA.
* Likely casperjs but there is already a python repo from QA.
   
   
==== Visual regression testing ====
==== Visual regression testing ====
1. Attach images with large changes to PRs.
* Attach images with large changes to PRs.


==== Spin up a docker instance with changes for a PR ====
==== Spin up a docker instance with changes for a PR ====
1. When you submit a PR, spin up an AWS and load marketplace-env at your SHA.
* When you submit a PR, spin up an AWS and load marketplace-env at your SHA.
2. This will aid manual testing (label:keep-test-env).
* This will aid manual testing (label:keep-test-env).
3. This will allow the integration tests to run against your PR.
* This will allow the integration tests to run against your PR.
4. Should this be app specific? what if zamboni updates while your PR is open?
* Should this be app specific? what if zamboni updates while your PR is open?


Video/screenshot of failing UI tests on Travis.
Video/screenshot of failing UI tests on Travis.
Confirmed users
1,158

edits