quantization
quantization copied to clipboard
Re-quantizing a model not possible?
Now that Xcode 10 is GM, I want to try 8bit quantized models. The issue is that my models have already been quantized to 16bit, and when I run the quantization script on them again, their size doesn't change. It does work if I take the original model and quantize it straight to 8bit though, so I'm scratching my head on this.
Original =>6.7 MB Original -> 16bit => 3.4 MB Original -> 8bit => 1.7 MB Original -> 16bit -> 8bit : 3.4 MB