lighteval
lighteval copied to clipboard
[FT] Pass reasoning effort to `litellm` endpoint.
Issue encountered
There is currently no way to pass the reasoning_effort parameter to litellm.completion(...). Without being able to turn on reasoning there is no way to do most flagship models justice. This might be a more general issue with other entrypoints too, am not familiar. LiteLLM supports optional literal values minimal, low, medium, high.
Solution/Feature
Would like to be able to specify something like
model_config = LiteLLMModelConfig(
model_name=model_name,
api_key=api_key,
generation_parameters=GenerationParameters(
reasoning_effort="high"
)
)
Possible alternatives
Perhaps adding it through GenerationParameters doesn't make sense, in which case it could be added to LiteLLMModelConfig directly.
hey ! thanks for raising the issue. it is in fact a generation parameter. feel free to open a PR with this and we will review asap :)