wweyl

从本地加载量化，程序没反应

3

`import torch from transformers import AutoModelForCausalLM, AutoTokenizer from transformers.generation.utils import GenerationConfig tokenizer = AutoTokenizer.from_pretrained("/root/autodl-tmp/model/Baichuan-13B-Base", use_fast=False, trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("/root/autodl-tmp/model/Baichuan-13B-Base", torch_dtype=torch.float16, trust_remote_code=True) model = model.quantize(8).cuda()` 从本地加载，再量化，程序没反应，内存也不涨。不知道哪些写的有问题