BitNet icon indicating copy to clipboard operation
BitNet copied to clipboard

pre-tuned kernel not found for model microsoft/BitNet-b1.58-2B-4T

Open grctest opened this issue 9 months ago • 0 comments

Using the latest version, I'm trying to setup BitNet following the setup_env.py help instructions:

python .\setup_env.py -help
usage: setup_env.py [-h]
                    [--hf-repo {1bitLLM/bitnet_b1_58-large,1bitLLM/bitnet_b1_58-3B,HF1BitLLM/Llama3-8B-1.58-100B-tokens,tiiuae/Falcon3-7B-Instruct-1.58bit,tiiuae/Falcon3-7B-1.58bit,tiiuae/Falcon3-10B-Instruct-1.58bit,tiiuae/Falcon3-10B-1.58bit,tiiuae/Falcon3-3B-Instruct-1.58bit,tiiuae/Falcon3-3B-1.58bit,tiiuae/Falcon3-1B-Instruct-1.58bit,microsoft/BitNet-b1.58-2B-4T}]
                    [--model-dir MODEL_DIR] [--log-dir LOG_DIR] [--quant-type {i2_s,tl2}] [--quant-embd] [--use-pretuned]

Running the following command ran into an error however:

python setup_env.py --hf-repo microsoft/BitNet-b1.58-2B-4T -q i2_s --use-pretuned
ERROR:root:Pretuned kernels not found for model microsoft/BitNet-b1.58-2B-4T

It seems that the microsoft/ was not removed, and so didn't match with "BitNet-b1.58-2B-4T"

grctest avatar Apr 25 '25 13:04 grctest