bitsandbytes
bitsandbytes copied to clipboard
lib.cquantize_blockwise_fp32 mutates absmax
System Info
OS: Ubuntu 22.04.5 LTS Python version: 3.11.11 GPU: A100-80GB Driver version: 565.57.01 CUDA version: 12.7 bitsandbytes version: 0.45.5
Reproduction
a = torch.tensor([i / 10 for i in range(10)])
b = torch.tensor([i / 10 for i in range(10)])
functional.quantize_blockwise(a, absmax=b)
after that
b = tensor([0.9000, 0.1000, 0.2000, 0.3000, 0.4000, 0.5000, 0.6000, 0.7000, 0.8000, 0.9000])
Expected behavior
This should not mutate absmax (imo)