BitNet

BitNet copied to clipboard

Fix #161: Modify decision equation of memory allocation size

4

This modification is a solution for wrong size memory allocation.(#143)

y-vectorfield

Fix #143: Add data type reinterpreting cast methods

4

This modification is a solution for some data type inconsistencies.(#161)

y-vectorfield

1. In the ggml_compute_forward_mul_mat() function in ggml.c, ggml_qgemm_lut() is executed first, which I think is an accumulation operation. 1. Subsequently, in the ggml_compute_forward_mul_mat_one_chunk() function, ggml_vec_dot_i2_i8_s() is executed, which performs a...

WanRui37

[New Bitnet Model Support Request] Deepgrove model Bonsai 0.5B - Add Channel Scales

1

A new SOTA bitnet model, Bonsai 0.5B, has come out. Seems to outperform larger bitnet models like Falcon 1B, 3B, TriLM 700M. Seems like they are going to release a...

zephrus9

ARM TL1 memory error: "double free or corruption (!prev)" and "free(): invalid next size (normal)"

2

I encountered this on Orange Pi 5 Plus (RK3588) with Ubuntu 22.04 LTS. The compilation environments (in a Conda env) are: - Python 3.9.21 - CMake 3.31.2 - Clang 18.1.8...

xxxxyu

Thre are some declarations of build mode in the `setup_env.py` scritpt. Hence, I suggest adding the following code in it. # Additional argument ## Target: https://github.com/microsoft/BitNet/blob/caf17ec438808bcf9b3da012216c8624a29b8cdc/setup_env.py#L205 ```python parser.add_argument("--build-mode", type=str, choices=["Release",...

y-vectorfield

It was impossible to build the sources of BitNet.cpp using the latest version of clang and clang++.

2

We built the sources of BitNet.cpp using clang. Now, we could not build the sources of it using clan version > 18. According to the README, we can install clang...

y-vectorfield

在preset_kernels目录下，bitnet-lut-kernels-tl1.h中，代码是否有错？int16x8_t和int8x16_t 两个数是否可以直接相加？具体含义是什么？

2

int16x8_t vec_c[4]; int8x16x2_t vec_v_left_0 = vzipq_s8(vec_v_0_left_tmp1, vec_v_0_left_tmp0); int8x16x2_t vec_v_right_0 = vzipq_s8(vec_v_0_right_tmp1, vec_v_0_right_tmp0); vec_c[0] += vec_v_left_0.val[0]; vec_c[0] += vec_v_right_0.val[0]; vec_c[1] += vec_v_left_0.val[1]; vec_c[1] += vec_v_right_0.val[1];

dujifeng

When I built the source codes of BitNet.cpp using gcc, some data type inconsistencies error occured. However, this error did not occur when I used clang. I investigated the root...

y-vectorfield

On Android device, compilation fails.

2

I installed the Debian system on my Android phone using Termux, and then followed the instructions step by step until I ran “python3 setup_env.py -md models/Falcon3-7B-Instruct-1.58bit -q i2_s” the following...

minne100