FlagEmbedding icon indicating copy to clipboard operation
FlagEmbedding copied to clipboard
verified publisher icon FlagOpen

BAAI/bge-reranker-v2.5-gemma2-lightweight 需要多少G的显存才能跑起来？

Open Hkaisense opened this issue 1 year ago • 2 comments

3090报内存不够？有大师试过吗？

Hkaisense avatar Aug 15 '24 08:08 Hkaisense

全量大概需要30G左右的GPU memory 可以在加载AutoModelForCausalLM.from_pretrained的时候引入参数torch_dtype=torch.float16，这样大概需要11G的GPU memory就可以加载了

545999961 avatar Aug 16 '24 08:08 545999961

A100 试了下没问题

ycjcl868 avatar Aug 19 '24 12:08 ycjcl868