phpunit
phpunit copied to clipboard
Move execution reordering to main iterator
Move all existing test execution reordering functionality to the code that answers the question what test shall we run next?
This refactoring is a continuation of the work for the recent @depends improvements and does not change anything for the end-users. The deliverable is a small pull request with @internal clean up.
Why
Folding the existing reordering mechanism into the core iterator and cleaning up the supporting code has tangible benefits:
- Reordering currently only happens on startup, giving the sorter a limited view. This is closely related to how
@dataProvidertests are loaded. - The
TestSuiteSortersorts the internal tests graph which can have side-effects. In contrast the extended iterator will simply loop over the existing tree in a different order. This makes it more flexible and robust. - Lay the groundwork for a clean implementation of on-demand
@dataProvidertests with properrewind/nexthooks. - The refreshed code is easier to understand and maintain. The
TestSuiteSorterhas too much knowledge of the runner internals.
To do
- [x] create an additional minimal
interfacefor tests that implement the requirements for the PHPUnit@depends-resolver- [x] implement
ReorderableTest(name suggestions welcome!) - [x] implement
getInternalId()for all PHPUnit test objects - [x] further improve
provides-requiresdependency resolver for all reordarable test types
- [x] implement
- [ ] use
TestSuiteIteratorfor all test iteration- [ ] extend iterator to keep to-do/done lists instead of just looping over a list
- [ ] sort a
TestSuiteduringIterator::rewind() - [ ] add dependency checks to
Iterator::next() - [ ] inject the result cache (see also:
--filtermechanism) - [ ] inject test ordering options
- [ ] retire
TestSuiteSorteras the main reordering mechanism- [ ] keep
TestSuiteSortercomparators for sorting individualTestSuites - [x] ~move test graph traversal for calculation original execution order~ utility function fits here
- [x] retire internal
getSorterUIDin favour of new interface - [ ] give unit tests for
TestSuite[Iterator|Sorter]a good wash
- [ ] keep
- [ ] update internal test naming scheme in result cache
- [ ] get code to LTS release quality
- [ ] full feature coverage
- [ ] project coverage, especially remove obsolete reordering guards