replicate-elixir icon indicating copy to clipboard operation
replicate-elixir copied to clipboard

Timeout for synchronous prediction

Open Doerge opened this issue 1 year ago • 0 comments

Sometimes models get's stuck in kind of infinite loops, taking way longer than normally. It would be great to have the following API:

ReplicateAI.run(..., timeout: :timer.minutes(2))

that would:

  • Submit a request.
  • Wait and check for a response.
  • Cancel the prediction if the timeout is reached, and return a timeout error.

Doerge avatar May 18 '24 13:05 Doerge