XProduct/Reading List/Testing Notes
V1 of project: Storing reading list content on server, fetching and processing on client Production by end of Q1, likely April/May First tagged version deploy by march 6, three weeks for travis team to deploy Not expecting real traffic until April, low requests per second
V2 of project (likely q2): Server side fetching, more load on servers, scaling - shared library for all clients to transform html to text
Does the project need/have:
Unit Tests
- 90% coverage
Smoke Tests
- light loadsv1 test
Regression Tests
Integration Tests
e2e Tests
- client teams doing testing
- stefan arentz - ios
- margaret lebo and richard newman - android (doing functional tests)
- gavin sharp - desktop (Desktop test plans/docs are here.)
Load Tests
- look at expected android numbers, load test to ensure that is doable (talk with android team)
- heavy load testing will be needed in V2 of reading list
Performance Tests
How are deployments done?
- will follow the standard jenkins deploy by mid-late march
- bob or dean likely handling from ops
Does the project automatically execute the required test types on checkin via a build server like jenkins or travis?
- yes, travis. jenkins build for staging should include some e2e tests (possibly from one of the client teams, or roll our own)
Can/should each of the test types fail the build on failed runs?
- yes, unit and smoke/load in travis, jenkins should include those + e2e + heavier load test when applicable
Are the test results visible in treeherder?
- no just in travis and jenkins
Next Steps
- Acceptance test plan for v1
- Load test v1
- Follow up with client teams to see what exactly they are testing e2e and if there is anything there we can leverage or help with (need to include e2e in jenkins builds)
- determine if any of these should be in treeherder
- if they aren’t doing e2e, what can we do to get this done
- Plan heavier load test and prep for next version, get tests in jenkins when ready
