how to make it work on 2 11G 2080ti?

[BEILOP]

how to change this code to make it work on 2 11G 2080ti?:

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("bigcode/starcoder")
# for fp16 replace with  `load_in_8bit=True` with   `torch_dtype=torch.float16`
model = AutoModelForCausalLM.from_pretrained("bigcode/starcoder", device_map="auto", load_in_8bit=True)
print(f"Memory footprint: {model.get_memory_footprint() / 1e6:.2f} MB")

[ArmelRandy]

You should try to update transformers (>= 4.31.0.dev0) , accelerate (>=0.21.0.dev0) and bitsandbytes. And instead of loading in 8 bit, try to use

model = AutoModelForCausalLM.from_pretrained("bigcode/starcoder", device_map="auto", load_in_4bit=True)