Shaoguang Mao
Shaoguang Mao
We will consider it. Thanks!
We will consider it. We also encourage you contribute to this feature. Feel free to make a PR once completed.
Unfortunately, no. If a model's weight parameters are not natively ternary, using the conversion function will result in the loss of weight values, leading to inaccurate results. We encourage more...
Could you please provide more details? Which command is extremely slow?
It is probably because the memory exhuasted. Can you provide the device information. You can also try to run the 700M model to check whether it works.
These models (available on huggingface ) are neither trained nor released by Microsoft. The tested models are used in a research context to demonstrate the inference performance of bitnet.cpp.