Fix galore lr display with schedulers

[vasqu]

What does this PR do?

See #31707 for a detailed rundown. Fixes #31707

Tl;dr: Galore still has issues displaying the correct lr due to the lr scheduler this time.

Before submitting

• [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• [x] Did you read the contributor guideline, Pull Request section?
• [ ] Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case.
• [ ] Did you make sure to update the documentation with your changes? Here are the documentation guidelines, and here are tips on formatting docstrings.
• [ ] Did you write any new necessary tests?

Who can review?

@muellerzr @SunMarc @amyeroberts @Minami-su

[vasqu]

Failing tests seem unrelated to me: TF and hub issues.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[vasqu]

@muellerzr Wouldn't it make more sense over here? https://github.com/huggingface/transformers/blob/3345ae733b6f4aeb7204a0f3e646a3cdbaad0023/tests/trainer/test_trainer.py#L1452

I would add two tests:

• A basic learning rate check without a scheduler.
• Cosine with warmup steps and check the lr logs to roughly follow the correct pattern.

Is that reasonable? I'm not sure when I'll have the time tho.

[vasqu]

@amyeroberts One timeout didn't make it through. Is it just my luck? :laughing:

[amyeroberts]

@vasqu Just bad luck - although we'll need to look into it on our side why these flaky failures are happening. Thankfully some re-runs worked. Thanks for your patience!