unbounded

Unit test for quantization functions

4

Use `ggml_internal_get_quantize_fn` to loop through all quantization formats and run sanity checks on the implemented functions. They are run by ctest, but also accept a few command line parameters for...

Use full range for q4_0 quantization

26

By preserving the sign of the highest magnitude value, we can make sure the highest value maps to -8 in our [-8, 7] range, which is currently unused. This is...

Continuous layouts for quantization q4_0c

9

Adds a q4_0c type that corresponds to the q4_0 layout but with a different memory layout. Draft status, currently only accelerated for AVX-512, will add a PoC of Neon acceleration...

Add implementation for user unbounded - SIMD city

1

Implementation that uses the Vector API for the following - scan for separators - calculate hash - n-way lookup in hash table - parse digits Not necessarily all beneficial. Somewhat...