Better DXR Testing

From MozillaWiki
Jump to: navigation, search

Here are some thoughts about improving the automated testing story for DXR.

Problems to solve

Things to keep

  • It's easy to run a test case as a regular DXR deployment so you can explore.
  • The top-level `make test` entrypoint is a nice, generic handle for Jenkins, no matter what language(s) or tool(s) the tests end up written with.

The Brave, New World

Have one class that supports running tests against a concrete DXR instance—files and all. That'll be good for complex tests with many files where the FS is the least confusing place to express them. (This now exists in DxrInstanceTestCase.)

Another class can support simple tests that need a single file or so. (Fortunately, everything ends up as a single file in C anyway.) You'll provide code as a string that gets laid down in a dynamically generated instance and compiled. There'll be a flag or well-marked place to put a breakpoint to leave the instance around in case you want to interact with it manually. We'll add a helper of some kind to make it easy to assert not only that certain files turn up in a search but that highlight ranges are as expected. This should make tests trivial to write and solve the coupling problem. The downside is that tests will run slower, since there will be more build runs. We can probably at least apply nose's multiprocessor stuff to it if we keep the temp files separate. (This now exists in SingleFileTestCase.)