GlasslessPizza

> Out of curiosity, did the resulting gguf sizes also change? Not enough to justify the difference: ``` b2037 31-1-2024 : 18,308,777,920 bytes b???? 25-2-2024 : 18,307,082,176 bytes b2329 03-3-2024...

> Can you try just before #5829? Sure, that would be b2314: ``` b2037 | 31-jan-2024 | 4.7009 +/- 0.02569 | 18,308,777,920 bytes b???? | 25-feb-2024 | 4.7249 +/- 0.02576...

I noticed that there are no fluctuations on other quantization types (such as Q3_K_M, Q4_K_S or Q4_0) but there are some variations on smaller non-mixtral models, so I tested a...

I managed to finish the mixtral test after a week of effort. Here's the result using always the same imatrix for all versions (same as in my OP). The only...

I tried b2699 hoping the regressions were fixed along the way: ikawrakow imatrix: ``` b2436 | 4.9467 +/- 0.04839 | 18240407488 bytes b2699 | 4.9473 +/- 0.04839 | 18240407488 bytes...

The problem is pretty much still present. I tried on b3334 today: ikawrakow imatrix: ``` b3334 | 4.9494 +/- 0.04843 | 18240407648 bytes ``` Recalculated imatrix: ``` b3334 | 5.3557...

I guess I'll bump this issue every two weeks to prevent the bot from autoclosing it, this is my life now. Tried on b3484 today: ikawrakow imatrix: ``` b3484 |...

> I have added the bug tag that will prevent the bot from closing the issue. Pointing at the specific PRs that introduced a regression would improve the chances of...

In order to fix the imatrix creation i had to recreate the base q8 from the original repo using a new llama.cpp version: ``` b3680 | 4.9383 +/- 0.04829 |...

The issue persists, tried today with b4550.