test-class icon indicating copy to clipboard operation
test-class copied to clipboard

Sort the list of test classes.

Open skington opened this issue 13 years ago • 3 comments

It's currently returned in essentially random order, which is less than helpful. If you have a large number of test classes, it helps for similar tests (e.g. Foo::Create, Foo::Delete, Foo::Modify, Foo::Update) to be run at similar times, so you can spot that all your test failures are to do with Foo::*, and cancel a test run once you spot a pattern, rather than those class being interspersed higgledy-piggledy with other tests involving Bar, Baz,Bletch and so on.

I suspect that mro::get_isarev is internally grovelling through a Perl hash, in which case the supposedly-random order will in fact be the same every time for a given version of Perl, so it's not even a useful random order.

We're currently applying the equivalent of this patch at $WORK, via monkey-patching Test::Class because you can't actually override methods via inheritance. It would be nice for this to make the core module.

skington avatar Feb 20 '13 18:02 skington

I suspect that mro::get_isarev is internally grovelling through a Perl hash, in which case the supposedly-random order will in fact be the same every time for a given version of Perl, so it's not even a useful random order.

Not true as of perl 5.18 - now hashes are truly randomized.

I see randomized tests as more of a feature than a liability -- it helps find unintended side effects of running tests.

This should instead be an option, with the default being the current (random) ordering.

karenetheridge avatar May 07 '14 21:05 karenetheridge

Now that hash order is truly random in modern Perls, yes, it has some use. I'll look into the most idiomatic Test::Class way of making this an option.

skington avatar May 07 '14 23:05 skington

BTW the caveats in https://github.com/adrianh/test-class/pull/10 apply here too -- big feature changes are not likely to be accepted (at least, not by me).

karenetheridge avatar May 07 '14 23:05 karenetheridge