Cook
Cook copied to clipboard
Benchmark time to schedule a workload
This will give us an idea of how long it should take to start some number of jobs, of various sizes.
The motivation is to understand how long it should take to launch a Spark cluster, so that we can figure out how multitenancy affects this, and if something special is needed.
There are 3 ways to do this:
- On a real big cluster (this is most accurate, but very hard)
- Within the simulator (this would require making the APIs match the offer pipeline and simulating it in realtime)
- Writing a model and testing with that (this is least accurate, but probably the easiest approach)