George D. Torres
George D. Torres
Agree this would be useful. Is this equivalent to caching every task within the workflow (assuming tasks are deterministic)?
@adriangonz Thanks! I added a test that passes `None`, `-1` and `cpu` for the device to ensure they all load into cpu.
@adriangonz :)
@sakoush if you don't mind taking a look, much appreciated!
Does anyone have any recommendations for alternative frameworks that allow per-model user-provided code like torchserve's `handler.py`?
One open question is: if the user doesn't pass `unknown_tool_retries`, should it default to `retries`? Or default to 1? Currently it is set to default to 1 so as not...
@DouweM Yes you're right; it will have to hallucinate a tool name twice for this to happen. So your suggestion is to not expose this as a configurable setting and...
@DouweM done!
@DouweM resolved your comments