chronos icon indicating copy to clipboard operation
chronos copied to clipboard

Spark jobs priority with Chronos

Open DatPhanTien opened this issue 7 years ago • 0 comments

Dear,

We are interesting in running Spark jobs using Chronos. There are two type of Spark jobs in our cluster.

  1. Batch jobs that have high workload but do not require fast response.
  2. Interactive jobs, that are triggered from user side, which have strict response time requirement (order of seconds or less).

We would like to know whether it is possible to put priority for such jobs, so that the interactive jobs have higher priority? Once the interactive jobs are triggered, batch jobs are supposed to quickly free resources (e.g., by getting killed) so that the interactive jobs can have the maximum resources allocated in order to meet the response time requirement.

So far, we could not find any documentation about Chronos that mentions this. Could you please enlighten us.

Best

DatPhanTien avatar Jun 20 '18 14:06 DatPhanTien