RiverDong

Cannot use int8

1

I tried to use 8xA100 to run BLOOM. But I cannot do load_in_8bit. I tried to follow the instruction [here](https://gist.github.com/younesbelkada/073f0b7902cbed2cbff662996a74162e) load the model by `model = AutoModelForCausalLM.from_pretrained(model_name, device_map='auto', load_in_8bit=True, max_memory=max_memory)`...

Unable to install annoy which is a dependency of capreolus

1

Hi, I am trying to run the code. And I need to install annoy which is a depedency of capreolus. However, I am unable to install it for reasons I...