CloudServices/Basic Service Testing: Difference between revisions

1. Unit tests -- run on every commit -- dev responsibility<br />

These should only touch a single module, and should be referentially transparent.<br />

They are stored alongside the source code, and should change at the same time.<br />

They can use fixtures, but should generally not use mocks.

2. Integration tests -- run on every 'build' -- dev/qa shared<br />

These tests should bridge more than one module (exactly two where practical)<br />

They should live in a tests directory in the source repository.<br />

If you have mocks for external components, this is where they should live.<br />

Depending on the framework, these may or may not require a 'deploy' to run. If possible they should not.<br />

3. End-to-end testing (also called verification testing or acceptance testing)-- run on every 'deploy' -- qa/ops shared<br />

In staging environments, qa should run these.<br />

In production environments, ops will run them.<br />

Ideally, no mocks here; staging testing and production testing should factor out all differences between them into parameters. This is important: both sets of tests should use the exact same scripts.

4. Monitoring (also called heartbeat testing) -- run continuously -- dev/ops responsibility<br />

Custom endpoints are a possibility here (but watch out for security implications)<br />

The tricky thing here is that the devs have to build testing functionality into the server-side, so as to make the ops scripts to access the testing as simple as possible.

Glossary of terms: