Sean comments

Repositories
Issues
Comments

Results 4 comments of


                                            Sean

How to generate <think></think> block in response

Yes, I second this, in OpenRLHF the special tokens are provided to the custom reward function. I had a real problem that my model didn't know to generate EOS token...

KD (distillation) chat template error when using pre-collected logprobs

Oh I see what happened here: V2 inherited V1, but the super() is supposed to be calling the function from `ChatTemplateStrategy`, but instead, it called the function from the parent...

Chat template or special tokens are modified during SFT training

My issue was actually my bad, I did not prepare data as "a list of dicts": https://openrlhf.readthedocs.io/en/latest/quick_start.html#prepare-datasets

Support NVIDIA cuOpt solver

Thanks! I'll close this.