Zhaohan Dong
Zhaohan Dong
Should probably add a dummy for testing if it looks good.
😅 If I'm allowed to modify the `LM` class, I'd say this part from Line 38 can be extracted to another function: ``` if self.model_type == "chat": completion = cached_litellm_completion...
@okhat I feel like letting the litellm inference functions conditionally use routers would actually make it cleaner. It is possible since the litellm.Router can be hashed for lru_cache. Let me...
Down to work on this if needed. @okhat maybe another function in dspy.LM similar to litellm_completion? Can also do a router kwarg to pass in a litellm.Router object.
Exactly what I hoped to ascertain I think litellm tries to have similar signature for Router as the plain generation methods. So I'm thinking it could be progressively do dependency...
Awesome! Maybe RoutedLM as name?
@denisergashbaev I tried a PR here: https://github.com/stanfordnlp/dspy/pull/1611. Dunno if you could collab?
@ryanh-ai Have a branch here https://github.com/zhaohan-dong/dspy/tree/litellm-router
New update here from the working group? https://httpwg.org/http-extensions/draft-ietf-httpbis-safe-method-w-body.html I can try to get onto it over the weekend.