CloudServices/Basic Service Testing

Different levels and frequencies of testing, in the context of web services:

Unit tests -- run on every commit -- dev responsibility

These should only touch a single module, and should be referentially transparent.
They are stored alongside the source code, and should change at the same time.
They can use fixtures, but should generally not use mocks.

Integration tests -- run on every 'build' -- dev/qa shared

These tests should bridge more than one module (exactly two where practical)
They should live in a tests directory in the source repository.
If you have mocks for external components, this is where they should live.
Depending on the framework, these may or may not require a 'deploy' to run. If possible they should not.

End-to-end testing (also called verification testing or acceptance testing) -- run on every 'deploy' -- qa/ops shared

In staging environments, qa should run these.
In production environments, ops will run them.
Ideally, no mocks here; staging testing and production testing should factor out all differences between them into parameters. This is important: both sets of tests should use the exact same scripts.

Monitoring (also called heartbeat testing) -- run continuously -- dev/ops responsibility

Custom endpoints are a possibility here (but watch out for security implications)
The tricky thing here is that the devs have to build testing functionality into the server-side, so as to make the ops scripts to access the testing as simple as possible.

Glossary of terms

module -- a minimally testable functional unit of code.
build -- compile but don't necessarily package
deploy -- package, install, make talk to the network.