Canmove, confirm
2,041
edits
Changes
→The Problem
Commit integration automation is incomplete and shallow, trading expedience for coverage, and is generally written from a developer's perspective to ensure the behavior of what they did write--which might not actually meet the requirements of what they should have written. Since the tests tend to be written against a particular section of the SUT, broad interactions may go uncovered. Large portions of the SUT may go untested because they necessarily rely on unreliable externals. As such, it frequently falls short of the needs of acceptance.
Conversely, build acceptance automation is too slow, device-bound, and sometimes has spurious failures due to non-determinism inherent in the SUTand its externals. As the user scenarios it tests often bridge multiple parts of the system, it's very fragile with poor isolation. The need to rely on user-like behavior and check after every step slows it down. The necessary scope means it often can't be written adequately until entire subsystems are in place. So it's often wholly unsuitable for the incremental growth, quick land-or-not decisions, and debugging assistance that a good continuous integration suite provides.
However, these are weaknesses in perception. When used for their own specific purposes, each type of automation provides great value. Further, the compromises each makes are compensated for by the other.
