Add a way to easily enable/disable thinking mode for reasoning models

[MaxLSB]

Issue encountered

When evaluating reasoning capable models (e.g., Qwen3, etc) with lighteval, there's currently no way to toggle enable_thinking directly via CLI or config.

Is there a way to work around this or does it require a PR ?

Solution/Feature

Maybe add a flag or config option to enable/disable thinking mode (i.e., pass enable_thinking=False to the tokenizer) for models that support it.

[berserkr]

Ditto on this!

[mohit-at-delhivery]

Hi team, if this has been implemented please release this or suggest the workaround for this. Such features are expected in such a popular library like lighteval. Kindly let us know how to disable thinking in qwen 3 in lighteval. Thank you.