BitNet-Transformers icon indicating copy to clipboard operation
BitNet-Transformers copied to clipboard
verified publisher icon Beomi

Great work. quick question on roadmap ETA?

Open darkman111a opened this issue 2 years ago • 4 comments

Just wondering when you were planning on implementing BitLinear layer to use 1-bit weights and custom cuda kernel for 1-bit weight? super thirsty for the code ha. Appreciate you.

darkman111a avatar Oct 26 '23 15:10 darkman111a

Currently I'm investigating on it. plan to make it compatible with Bitsandbytes. Since I'm not familiar with CUDA or C++, it'll takes some time, so currently I could not guarantee the ETA. 😂

Beomi avatar Oct 27 '23 13:10 Beomi

you may want to check out https://github.com/IST-DASLab/qmoe. They created some custom cuda functions for sub 1-bit weights.

DewEfresh avatar Oct 27 '23 13:10 DewEfresh

you way want to check out https://github.com/IST-DASLab/qmoe. They created some custom cuda functions for sub 1-bit weights.

Thanks for information! I'm looking on it 😉

Beomi avatar Oct 27 '23 13:10 Beomi

Appreciate you. Stay Blessed! This might assist you as well -> https://github.com/brycexu/BNN_Kernel

darkman111a avatar Oct 28 '23 15:10 darkman111a