llama2.zig icon indicating copy to clipboard operation
llama2.zig copied to clipboard
verified publisher icon clebert

Add support for 8-bit Quantization

Open clebert opened this issue 2 years ago • 0 comments

See:

  • https://github.com/karpathy/llama2.c/issues/277
  • https://github.com/karpathy/llama2.c/pull/298
  • https://github.com/karpathy/llama2.c/pull/312
  • https://github.com/karpathy/llama2.c/pull/364
  • https://github.com/ggerganov/llama.cpp/issues/397
  • https://arxiv.org/pdf/2101.01321v3.pdf

clebert avatar Oct 19 '23 16:10 clebert