ultoris comments

Repositories
Issues
Comments

Results 3 comments of


                                            ultoris

≈65% speedup of the AVX-512 implementation of `ggml_vec_dot_q4_0()`

Test on Raptor Lake fails as the Intel has dropped support for AVX512 in latest generations of consumer CPUs. It supports avx_vnni though (256bit instructions instead of 512bit in AVX512),...

≈65% speedup of the AVX-512 implementation of `ggml_vec_dot_q4_0()`

> I don't have access to any AVX-VNNI-enabled hardware, though, so I can't test/benchmark it. I can make a separate PR if you're willing to help testing it. Sure. I...

Model request: OpenAI's gpt-oss-20b model

SYCL is about 20% slower than pure CPU on my system (Intel Core Ultra 155H). With IPEX token generation was almost on par with pure CPU llama.cpp (memory bound) but...