qunitjs
Core: Create config.testIds to expose all available test ids
What this does
Currently each testId is calculated as each test is executed. Config.testIds pre-hashes all the loaded tests on demand and returns all testIds.
How is this helpful
Currently test runners can get access to test modules via config.modules and partition the modules based on some distribution strategy for running the tests in parallel. This is fine but it can lead to unbalanced distribution as some modules have a lot of tests and some modules are testing integration (longer running) test instead of unit test.
Partitioning the tests by modules caused some instance of test runners to run way longer than others as some heavier running modules might be clumped together in the same instance.
To mitigate and minimize the unbalanced test runs, the proposed solution is to partition based on each individual test with testId. The test runners can now access testIds beforehand and can distribute those ids with the testId url param. This will minimize the test execution time descrepencies between each individual instance of the test runners.
Does this pre-hashing match the existing unique hashes during the run? The existing hash mechanism modifies clashing test-names (white whitespace suffixes) to get unique IDs.
I want to ensure these pre- and post-ids match up, and if there's any way to avoid hashing the same thing twice (we're pre-allocating, so maybe the latter can reuse that cache and not recalculate?).
Does this pre-hashing match the existing unique hashes during the run? The existing hash mechanism modifies clashing test-names (white whitespace suffixes) to get unique IDs.
You are right, the new code did not account for that. I am going to abstract out the hashing collision management to a util and fix the code.
I want to ensure these pre- and post-ids match up, and if there's any way to avoid hashing the same thing twice (we're pre-allocating, so maybe the latter can reuse that cache and not recalculate?).
The hashing is incredibly fast, I had 10k tests and it returned the whole array of hashes immediately upon calling the getter. There's a storage tradeoff for caching for something that might not improve performance measurably.
Also, QUnit.config.testIds only evaluates on demand for certain cases that need the testIds in advance. I would argue not to add caching complexities.
