trl icon indicating copy to clipboard operation
trl copied to clipboard
verified publisher icon huggingface

Different with online dpo papers

Open mst272 opened this issue 1 year ago • 1 comments

I see that the paper says that the Annotator can be adjusted through prompt. But the implementation of trl is score. Is this different from the paper?

mst272 avatar Sep 05 '24 09:09 mst272

Indeed, it's different from the paper for now as we will soon implement Online DPO with judge (ie, LLM annotator). The PR will be linked to this issue.

qgallouedec avatar Sep 08 '24 15:09 qgallouedec