Alexia Jolicoeur-Martineau
Alexia Jolicoeur-Martineau
Hi Yaser, You can use the lines of code here https://github.com/AlexiaJM/score_sde_fast_sampling/blob/5da8f3fe103ee5ac3c3a336f16cc06c9541f0ed9/experiments.sh#L165 to compare the various different sampling methods on FFHQ-256. You'll want to replace ffhq_256.py with ffhq.py in the lines...
1. You need the real data to calculate the FID. Assuming you already have the FID and don't have eval.enable_loss=False, you could just comment out the line https://github.com/AlexiaJM/score_sde_fast_sampling/blob/5da8f3fe103ee5ac3c3a336f16cc06c9541f0ed9/run_lib.py#L234 and things...
It contains the metrics over time. I'm not sure why it would do that. 🤔 Check that the folders exists: /cluster/51/dichang/datasets/mcvd and /cluster/51/dichang/datasets/mcvd/smmnist_cat/log. Make sure that the ninja package is...
It seems like other people have had similar issues and they propose some solutions, see: https://github.com/mapillary/inplace_abn/issues/104 and https://github.com/mapillary/inplace_abn/issues/106#issuecomment-475460496. I really don't know what to do with ninja or even what...
Related to https://github.com/huggingface/alignment-handbook/issues/45
See https://github.com/huggingface/alignment-handbook/blob/87cc800498b17432cfb7f5acb5e9a79f15c867fc/scripts/run_dpo.py#L91 and https://github.com/huggingface/alignment-handbook/blob/87cc800498b17432cfb7f5acb5e9a79f15c867fc/scripts/run_sft.py#L98.
Problem is fixed by replacing: ``` raw_datasets = raw_datasets.map( apply_chat_template, fn_kwargs={"tokenizer": tokenizer, "task": "dpo"}, num_proc=data_args.preprocessing_num_workers, remove_columns=column_names, desc="Formatting comparisons with prompt template", ) ``` with ``` raw_datasets = raw_datasets.map( apply_chat_template, fn_kwargs={"tokenizer":...
Update: The problem is fixed by using the latest version of datasets (v2.17.1), so my above fix is not needed. So all you need to do is change the requirements...
This is how it should be, but it must be bugged in this old version of datasets. Yes, I tested it with prints inside the apply_chat_template. It doesn't print with...
@pankayaraj Did you find a solution? I have the same bug. Mine is at loss=0.6931 instead of .6932 and the logps/rejected goes down to crazy numbers near the end ('logps/rejected':...