lambdaisland
Add a `pre-test-report` hook
In #260, a user was trying to extend Kaocha to include details about any Postgres errors that happened during a test. They used pre-test to log the test start time and attempted to use post-test to add additional metadata to identify the end of test (allowing them to search the logs for events that happened between the start and end times of the test), but this wasn't working because the reporter would handle the error result before the post-test hook was called.
How this would work
This hook would happen immediately after the test runs and before the reporter runs, allowing for last-minute alterations to the testable map. I did some looking into wrap-run, and I don't think that hook covers this situation, but that should be verified before implementing this hook. In theory, if you're implementing your own reporter, it could run its own logic before actually doing any error reporting; however, this would only work for custom reporters, so I think this hook is useful.
For future contributors
Two design decisions:
- Whether we should call this hook for tests that fail to load or are pending. I'm inclined to say yes because the reporter runs in those cases.
- Whether we should call this hook for successful tests.
Actually adding this hook wouldn't be too tricky:
- [ ] Register a new hook in
hierarchy.clj - [ ] Call the hook in the
testable/run-testablefunction. - [ ] Add the hook to the documentation.
- [ ] Add some tests along the lines of our existing hook tests.
All that being said, I consider this a low-priority feature since you can often hack around it using an entirely separate custom reporter or overriding the default error reporter method.
