BitNet

BitNet copied to clipboard

FileNotFoundError: [Errno 2] No such file or directory: 'build/bin/llama-cli'

3

MacOS15.3.2 ,Apple M4 Max, can not run, all the logs no error, but when run the commands show the error python run_inference.py -m models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf -p "You are a helpful assistant"...

951336ggg

error loading model: PrefetchVirtualMemory unavailable

2

when run the command "python run_inference.py -m models/ggml-model-i2_s.gguf " an error occurs. it echo "llama_model_load: error loading model: PrefetchVirtualMemory unavailable", who know that why?

jefferthang

llama-cpp not working

4

Hello The readMe advises to Launch setup_env python script, that will recompile llama-cpp-cli exécutable. I want to easily integrate the optimized new llama-cpp in a gradio or streamlit python app:...

PrinceSajjadHussain

local bitnet model don't have the same quality as online bitnet

3

It seems, the bitnet local model doesn't have the same quality as the online demo from microsoft site : https://bitnet-demo.azurewebsites.net/ What are the parameters for online demo model ? i...

lbarasc

a demo sample problem

2

minkuan

Add Collab Notebook BitNet demo link to README.md

1

Following #181 - this is a small update adding [a Collab notebook link](https://colab.research.google.com/drive/1YWEISZD3OTr4L11g8-PK4l0H7dQXarUo#scrollTo=cIFDpd9TzsAM) to the README - which lets users install and explore BitNet without any local setup. Edit permissions...

neonwatty

setup_env.py error on MacBook M3 - FileNotFoundError: [Errno 2] No such file or directory: 'models/BitNet-b1.58-2B-4T/config.json'

4

Following directions to setup on MacBook Pro M3 ` python3 setup_env.py -md models/BitNet-b1.58-2B-4T -q tl1 INFO:root:Compiling the code using CMake. INFO:root:Loading model from directory models/BitNet-b1.58-2B-4T. INFO:root:Converting HF model to GGUF...

steve8210

How can I install on window11?

1

i tried to install bitnet on window11 with intel 13th i5 and i got this log ```msbuild 버전 17.13.19+0d9f5a35a(.NET Framework용) build_info.vcxproj -> C:\Users\User\Desktop\BitNet\build\3rdparty\llama.cpp\common\build_info.dir\Release\build_info.lib In file included from C:\Users\User\Desktop\BitNet\src\ggml-bitnet-lut.cpp:10: C:\Users\User\Desktop\BitNet\3rdparty\llama.cpp\ggml\src\..\..\..\..\include\bitnet-lut-kernels.h(1127,54): warning...

kharu217

enable parallel build

1

catzy007

why arm don't support tl2 kernel

3

Hi , can i know why arm don't support tl2 kernel? i guess the simd instruction no support? 

nigelzzz