zhuyang2356

I am running the controlnet nightly version, After I run a picture, the gpu memory has added about 2GB. Then I run another pictures, the gpu memory added another 2GB....

 the problems are above. My Torch Version is 2.0.1, Cuda Version is 11.8: My commands are as follows, #--------------Prepare the Environment source /root/miniconda3/bin/activate conda create -n geneface python=3.9 conda...

用2080TI 22G单张显卡跑 qwen 1.5 32B-INT4模型没有成功

5

总是报显存过小问题，设置了一些（比如 gpu_memory_utilzation, quantization）也没有用。请大伙帮忙看看 顺带提一下，2080TI 22G显卡是有22356MB的总显存。 #环境 ubuntu 20.04 cuda 12.1 pytorch 2.3.0+cu121 transformers 4.40.2 vllm 0.4.2 #代码如下： from transformers import AutoTokenizer from vllm import LLM, SamplingParams tokenizers=AutoTokenizer.from_pretrained("../QwenModel/qwen15/Qwen1.5-32B-Chat-GPTQ-Int4") sampling_params = SamplingParams(temperature=0.7,...