jacqueline he comments

Repositories
Issues
Comments

Results 3 comments of


                                            jacqueline he

ImportError: Using `bitsandbytes` 8-bit quantization requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes: `pip install -i https://pypi.org/simple/ bitsandbytes`

fwiw, downgrading to a lower version of transformers helped resolve the issue for me (4.38.2 to 4.31.0).

Different results on Bias-NLI evaluation dataset

Hi, I'm not able to get your results; I was able to get very close numbers to the ones reported in the Github repo just now (see screenshot with timestamps...

StereoSet benchmark for GPT2

Hi, thanks for your interest in our work! :) That’s a great question! GPT-2 is trained auto-regressively and therefore cannot be evaluated in the same manner as a masked language...