TensorRT icon indicating copy to clipboard operation
TensorRT copied to clipboard
verified publisher icon pytorch

❓ [Question] How to compile a model with A16W8?

Open jiangwei221 opened this issue 2 years ago • 1 comments

Hi Torch-TensorRT team:

I'm wondering how can I compile a model with 8 bit weights, but using 16 bit activations? Thanks a lot!

jiangwei221 avatar Jan 23 '24 12:01 jiangwei221

May I know how you've obtained this model (model source) ? Is it finetuned with any quantization technique ?

peri044 avatar Jan 25 '24 20:01 peri044