replicate-elixir
replicate-elixir copied to clipboard
Timeout for synchronous prediction
Sometimes models get's stuck in kind of infinite loops, taking way longer than normally. It would be great to have the following API:
ReplicateAI.run(..., timeout: :timer.minutes(2))
that would:
- Submit a request.
- Wait and check for a response.
- Cancel the prediction if the timeout is reached, and return a timeout error.