sorasoras
sorasoras
@kaneawk 我也尝试 在本地编译这个 项目 过不了 git submodule update --init --recursive error: no such remote ref 5d5914f00b8adefd383f66b8b352855ece22a1d7 Fetched in submodule path 'src/main/jni/libevent', but it did not contain 5d5914f00b8adefd383f66b8b352855ece22a1d7. Direct fetching of...
啊 Windows 10不行啊 我用 Windows git 去编译。 如果有binary 就好了 但是已经找不到了
拿个虚拟机试试咯
ArmV8 跑 AES速度快很多 而且省电 应该有不少帮助吧
There are some different between quant from BF16-FP32 to BF16-FP16. It's not the same model when compare PPL between FP16 and FP32, and it behave differently. It would be interest...
> I don't hold any demands on your time. In terms of resources, Mozilla is sponsoring me to help llama.cpp so you've got a lot more resources than before. At...
> Is there anything special needed to see performance gains? I cloned/built/tested this PR branch and am seeing no change in performance on CPU (CUDA support flags disabled at compile...
> > For CPU, I think you need something that support bf16 acceleration like AVX512VNNI? > > also, you need conversion script that just copy BF16 weight from py to...
I believe llama cpp does not support long rope which is use by 128k variant.
> Oh yep saw it! We might instead be adding LoRA+, which has similar results. Technically DoRA is already enabled in Unsloth, but just not optimized dora+qlora work already? I...