BitNet
BitNet copied to clipboard
kyegomez
Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch
model_id = "h2oai/h2o-danube-1.8b-chat"# 
**Describe the bug** Where to download bitnet model? Which is shown in this example code: ``` bitnet.load_model("../model_checkpoint.pth") # Download model ``` **To Reproduce** Execute example. No model loaded. **Expected behavior**...
Referencing this paper: https://arxiv.org/pdf/2310.11453.pdf Code part: https://github.com/kyegomez/BitNet/blob/984ec72c2a45a88b739c85668690fe1abbdf3152/bitnet/bitlinear.py In general, it seems that the code does not match the paper, mainly Equation (1), (4) and (11). It also seems to be...
Thanks for your quick implementation! I was reading through `bitnet/bitbnet_b158.py` and just had a short question. In your implementation of `quantize_weights` you use the same procedure as outlined in the...
**Describe the bug** A clear and concise description of what the bug is and what the main root cause error is. Test very thoroughly before submitting. **To Reproduce** Steps to...
**Describe the bug** After 5300 iteraitons loss near 2.7, is it still supposed to spit out near giberish? **To Reproduce** Running on CPU, macbookkair M2, omitting the model.cuda() line **Expected...
Thank you for sharing this incredible work! I speculate that it's an issue of library versions, but I'm receiving the following error when attempting to run unmodified train.py: `RuntimeError: The...
Hello @kyegomez In the inference code of huggingface_example.py, it appears that replace_hf is executed, followed immediately by inference. However, upon examining replace_hf.py, I noticed it converts linear layers to bitlinear...
**Describe the bug** readme.md contains a link to https://drive.google.com/file/d/1gBuZRFBqMV3cVD902LXA_hmZl4e0dLyY/view which reports "Sorry, the file you have requested does not exist." **To Reproduce** Click on link :) **Expected behavior** Link would...
**Describe the bug** Installation does not work with `pipenv install bitnet`, `pipenv run pip` install bitnet. **To Reproduce** Apple M2, MacOS: 14.3.1, Python(in pipenv environment): 3.12.1 Run `pipenv install bitnet`,...
