FasterTransformer icon indicating copy to clipboard operation
FasterTransformer copied to clipboard
verified publisher icon NVIDIA

MPT-7B model conversion?

Open SinanAkkoyun opened this issue 2 years ago • 2 comments

Hello! I'd like to know how to convert the standard MPT-7b model weights to the right format to run inference with?

SinanAkkoyun avatar May 09 '23 06:05 SinanAkkoyun

https://github.com/mosaicml/llm-foundry/pull/169

ankit-db avatar May 27 '23 18:05 ankit-db

FasterTransformer development has transitioned to TensorRT-LLM.

MPT is supported in TensorRT-LLM. Please take a try.

byshiue avatar Oct 20 '23 10:10 byshiue