nativelink icon indicating copy to clipboard operation
nativelink copied to clipboard

Actions with no listeners that are old should be removed from queue

Open allada opened this issue 2 years ago • 3 comments

If an action is old (set in config) and has no more listeners it should just be evicted from the queue, since no one is waiting for the action.

see: https://github.com/TraceMachina/turbo-cache/blob/f15146d0cf94e1a9159f14db775efe4a3e27355d/cas/scheduler/simple_scheduler.rs#L432

We should implement this using timeouts and upon timing out signal to the worker to kill the action.

allada avatar Oct 25 '23 02:10 allada

What exactly qualifies an action as old?

It doesn't make much sense to me to compare the timeout for an action with time elapsed since creation, so I think it has to be one of the following:

  1. time the worker has spent trying to complete the task (would have to be a new field I think)
  2. time since the action was loaded from CAS

zbirenbaum avatar Mar 02 '24 02:03 zbirenbaum

In this case, I think it should track how long the item has been in the queue with no clients waiting on it. Then based on the config we drop or terminate the job after X number of seconds.

allada avatar Mar 02 '24 05:03 allada

In this case, I think it should track how long the item has been in the queue with no clients waiting on it. Then based on the config we drop or terminate the job after X number of seconds.

Got it, thanks for the clarification!

zbirenbaum avatar Mar 02 '24 19:03 zbirenbaum

This has been fixed a few different ways.

allada avatar Sep 02 '24 23:09 allada