jltchiu

Update: I use the mt-bench master branch to run the benchmark on 3 models with gpt-4 zephyr-7b-sft-qlora(downloaded) 6.365625 zephyr-7b-dpo-qlora(downloaded) 4.443038 zephyr-7b-dpo-qlora(trained) 1.883648 Even the downloaded qlora dpo model is worse...