Add test-execution optimization and sharding options

[lukfor]

This PR addresses #199.

Optimizing Test Execution

--related-tests <files>

Finds and executes all related tests for the provided .nf or nf.test files. Multiple files can be provided space separated.

--follow-dependencies

When this flag is set, nf-test will traverse all dependencies when the related-tests flag is set. This option is particularly useful when you need to ensure that all dependent tests are executed, bypassing the firewall calculation process.

--only-changed

When enabled, this parameter instructs nf-test to execute tests only for files that have been modified within the current git working tree.

--changed-since <commit_hash|branch_name>

This parameter triggers the execution of tests related to changes made since the specifie commit. e.g. --changed-since HEAD^ for all changes between the HEAD and HEAD - 1.

--changed-until <commit_hash|branch_name>

This parameter initiates the execution of tests related to changes made until the specified commit hash.

--graph <filename>

Enables the export of the dependency graph as a dot file. The dot file format is commonly used for representing graphs in graphviz and other related software.

Sharding

This parameter allows users to divide the execution workload into manageable chunks, which can be useful for parallel or distributed processing.

--shard <shard>

Splits the execution into arbitrary chunks defined by the format i/n, where i denotes the index of the current chunk and n represents the total number of chunks. For instance, 2/5 executes the second chunk out of five.

--shard-strategy <strategy>

Description: Specifies the strategy used to build shards when the --shard parameter is utilized. Accepted values are round-robin or none.. This parameter determines the method employed to distribute workload chunks among available resources. With the round-robin strategy, shards are distributed evenly among resources in a cyclic manner. The none strategy implies that shards won't be distributed automatically, and it's up to the user to manage the assignment of shards. Default value is round-robin.

[sateeshperi]

Wow! some pretty great features coming with this PR I see 🚀

@lukfor FYI - Adam Talbot & Carson Miller were trying to implement some of the functionality here with a GHA

[lukfor]

Thanks for the info! We are in the final stage of preparing the new release, along with the preprint.

[adamrtalbot]

This is excellent. I would be happy to replace the Python internals of my GHA with this once it's available!