Discrepancy in rm_scores Length for Mistral-7B-SPPO Iter1 Dataset

[srzhu97]

I noticed data-mistral-7b-instruct-sppo-iter1 column rm_scores is a list length of 7 where as there are only 5 generated responses. data-mistral-7b-instruct-sppo-iter2 and data-mistral-7b-instruct-sppo-iter3 looks correct and both have length = 5

[MeckyWu]

We included the original two responses in the UltraFeedback dataset and the 5 generated responses in the first iteration's dataset.

This is not done in the released Github repo though. Our official code only generates responses at each iteration. To reproduce the results it is recommended to run the pipeline in the official code directly. @angelahzyuan