BitNet icon indicating copy to clipboard operation
BitNet copied to clipboard

fix for Microsoft's incompetence

Open gilbrotheraway opened this issue 9 months ago • 2 comments

export CC=clang
export CXX=clang++

 cmake -B build -DGGML_BITNET_ARM_TL1=ON -DCMAKE_C_COMPILER=clang-19 -DCMAKE_CXX_COMPILER=clang++-19

CC=/usr/bin/clang CXX=/usr/bin/clang++ python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s

gilbrotheraway avatar Apr 26 '25 02:04 gilbrotheraway

one wonders how they got 15k stars when ikarakrow llamacpp fork is 9 months old and runs much better

gilbrotheraway avatar Apr 26 '25 02:04 gilbrotheraway

comparison between this fork and ikawrakow, running on rk3588 SBC

llama_perf_context_print: prompt eval time =    9751.63 ms /    16 tokens (  609.48 ms per token,     1.64 tokens per second)
llama_perf_context_print:        eval time =    1379.77 ms /     9 runs   (  153.31 ms per token,     6.52 tokens per second)
llama_print_timings: prompt eval time =      90.99 ms /     6 tokens (   15.17 ms per token,    65.94 tokens per second)
llama_print_timings:        eval time =    4644.74 ms /   127 runs   (   36.57 ms per token,    27.34 tokens per second)

gilbrotheraway avatar Apr 26 '25 20:04 gilbrotheraway