Nexes the Elder

Refactor: GGUF my Repo tool on HF needs its scripts updated with the new naming scheme

1

### Background Description  On HF, GGUF my Repo scripts seem to currently use the old one, and LlamaCPP probably got...

Changes for the existing quant strategies / FTYPEs and new ones

2

Here's a few edits I consider useful to improve a bit the IQ2 model quant strategies for some models: - The tensor attn.v.weight passed in Q4_K for models like Gemma...

Quantize: specify each major tensor quant in CLI for common LLMs

13

This PR simply replicates the tensor per tensor custom quantization CLI feature brought by Ikawrakow for the token embeddings and output tensors in #6239 to : - attn_q.weight - attn_k.weight...

Hey. I would like to make a merge of Nemotron 3.1 on a Llama 3.3 instruct base. From what I understand, Nemotron 3.1 is based on Llama 3.1 instruct. And...

Some minor quant strategies tweaks

2

Here's what I'd suggest for starters : - Rationalize Q2_K_S ffn_down and attn_v (+1% size, -2.5% ppl) - Bump attn_v and attn_k for Q2_K_S and Q2_K if GQA>=2. Uncripple attn_k...