lighteval icon indicating copy to clipboard operation
lighteval copied to clipboard

[FT] Pass reasoning effort to `litellm` endpoint.

Open gardarjuto opened this issue 6 months ago • 1 comments

Issue encountered

There is currently no way to pass the reasoning_effort parameter to litellm.completion(...). Without being able to turn on reasoning there is no way to do most flagship models justice. This might be a more general issue with other entrypoints too, am not familiar. LiteLLM supports optional literal values minimal, low, medium, high.

Solution/Feature

Would like to be able to specify something like

model_config = LiteLLMModelConfig(
    model_name=model_name,
    api_key=api_key,
    generation_parameters=GenerationParameters(
        reasoning_effort="high"
    )
)

Possible alternatives

Perhaps adding it through GenerationParameters doesn't make sense, in which case it could be added to LiteLLMModelConfig directly.

gardarjuto avatar Aug 12 '25 13:08 gardarjuto

hey ! thanks for raising the issue. it is in fact a generation parameter. feel free to open a PR with this and we will review asap :)

NathanHB avatar Aug 12 '25 15:08 NathanHB