direct-preference-optimization icon indicating copy to clipboard operation
direct-preference-optimization copied to clipboard
verified publisher icon eric-mitchell

Reproducing Win Rate inference for TL;DR

Open jdchang1 opened this issue 2 years ago • 1 comments

Hi, I have been trying to reproduce the win rate results from the paper for summarization and I'm struggling to get similar values. I wonder if you've experienced this as well? Could this perhaps be due to changes made to GPT-4 since the published results?

Thank you!

jdchang1 avatar Jan 09 '24 18:01 jdchang1

Hi @jdchang1, may I ask how you achieved summarization task? Just change the dataset to TL;DR?

yurunsheng1 avatar Apr 18 '24 13:04 yurunsheng1