Allow specifying the required number of workers for a round

[aklenik]

Please share the technical limitation of Caliper that you encountered.

Caliper uses the same constant number of workers for each round. This complicates the implementation of "initialization workloads" that run before the actual benchmark workloads (but in the same Caliper execution).

The initialization (data ingestion) part of workloads could pose the following challenges:

1. Workers must ensure that they create mutually exclusive, unique entries
2. Entry creation could be non-trivial and hard to split among workers
3. Multiple workers (and their extra load rate) might not be required, a single or few workers might be sufficient to load the initial data into the SUT

Moreover, different rounds usually contain different workload logic, which might require different worker loads towards the SUT. Using the same number of workers for every round might be too restricting in some cases.

Please detail your feature idea that could alleviate the limitation.

Allow the specification of the required number of workers for each round, defaulting to the available number of workers, if not specified (ensuring backward compatibility).

Please share some details about your use case if possible, and how the new feature would make Caliper a better performance benchmarking framework.

The use case is the implementation of a benchmark that requires the insertion of many (~millions of) entries into the world state before the actual workload can be run. On one hand, this is a write-only/write-heavy operation on the smart contract side, and a single worker load is enough to saturate the SUT. On the other hand, the required workers for the actual benchmark run need to be scalable, while the init run requires substantially fewer workers (e.g., 100 vs 1).

All in all, allowing this configuration parameter to be specified for each round individually will increase the flexibility of Caliper.

Please share any suggestions about the new feature's code/configuration API (using formatted YAML segments or pseudo-code).

The proposed extension for the benchmark configuration file (requiredWorkers attribute):

test:
  workers:
    type: local
    number: 10
  rounds:
  - label: loading-phase
    requiredWorkers: 1
    # other round attributes
  - label: execution-phase
    # other round attributes, using the available 10 workers as default

On the implementation side, the manager service can signal only the workers for a new round whose index is below the required threshold (essentially lying about the total number of workers for that round).