phpunit icon indicating copy to clipboard operation
phpunit copied to clipboard

Move execution reordering to main iterator

Open epdenouden opened this issue 5 years ago • 0 comments

Move all existing test execution reordering functionality to the code that answers the question what test shall we run next?

This refactoring is a continuation of the work for the recent @depends improvements and does not change anything for the end-users. The deliverable is a small pull request with @internal clean up.

Why

Folding the existing reordering mechanism into the core iterator and cleaning up the supporting code has tangible benefits:

  • Reordering currently only happens on startup, giving the sorter a limited view. This is closely related to how @dataProvider tests are loaded.
  • The TestSuiteSorter sorts the internal tests graph which can have side-effects. In contrast the extended iterator will simply loop over the existing tree in a different order. This makes it more flexible and robust.
  • Lay the groundwork for a clean implementation of on-demand @dataProvider tests with proper rewind/next hooks.
  • The refreshed code is easier to understand and maintain. The TestSuiteSorter has too much knowledge of the runner internals.

To do

  • [x] create an additional minimal interface for tests that implement the requirements for the PHPUnit @depends-resolver
    • [x] implement ReorderableTest (name suggestions welcome!)
    • [x] implement getInternalId() for all PHPUnit test objects
    • [x] further improve provides-requires dependency resolver for all reordarable test types
  • [ ] use TestSuiteIterator for all test iteration
    • [ ] extend iterator to keep to-do/done lists instead of just looping over a list
    • [ ] sort a TestSuite during Iterator::rewind()
    • [ ] add dependency checks to Iterator::next()
    • [ ] inject the result cache (see also: --filter mechanism)
    • [ ] inject test ordering options
  • [ ] retire TestSuiteSorter as the main reordering mechanism
    • [ ] keep TestSuiteSorter comparators for sorting individual TestSuites
    • [x] ~move test graph traversal for calculation original execution order~ utility function fits here
    • [x] retire internal getSorterUID in favour of new interface
    • [ ] give unit tests for TestSuite[Iterator|Sorter] a good wash
  • [ ] update internal test naming scheme in result cache
  • [ ] get code to LTS release quality
    • [ ] full feature coverage
    • [ ] project coverage, especially remove obsolete reordering guards

epdenouden avatar Jun 17 '20 11:06 epdenouden