IntelAI
Quantizing the Official TF Resnet Model
I'm trying compare the accuracies between resnet and its quantized version. First, I downloaded the resnet_v1 saved_model and used tensorflow's freeze_graph tool to freeze the graph.
I then followed the guide here, all the way until the middle of step 6, until I got the model into the logging format (logged_quantized_graph.pb).
When I tried to run inferenece, however, the computation graph failed to load. I ran into this error:
tensorflow.python.framework.errors_impl.InvalidArgumentError: No OpKernel was registered to support Op 'QuantizedConv2D' used by node import/resnet_model/conv2d_1/Conv2D_eightbit_quantized_conv (defined at eval_resnet.py:471) with these attrs: [out_type=DT_QINT32, Tinput=DT_QUINT8, dilations=[1, 1, 1, 1], strides=[1, 1, 1, 1], Tfilter=DT_QINT8, padding="SAME"]
Registered devices: [CPU, XLA_CPU]
Registered kernels:
device='CPU'; Tinput in [DT_QUINT8]; Tfilter in [DT_QUINT8]; out_type in [DT_QINT32]
It seems like the graph has a qint8 filter while the registered kernels only support a quint8 filter for the quantizedconv2d layer. I found a related issue, and tried building tensorflow from source with the --config=mkl and --copt="-DINTEL_MKL_QUANTIZED" flags. Then I tried loading the graph again using this specially built version of tf.
However, now I run into this error which I can't seem to resolve:
2019-09-09 11:14:59.945553: F tensorflow/core/graph/mkl_layout_pass.cc:3459] Non-OK-status: ret_status status: Invalid argument: NodeDef mentions attr 'narrow_range' not in Op<name=_MklQuantizeV2; signature=input:float, min_range:float, max_range:float, mkl_input:uint8, mkl_min_range:uint8, mkl_max_range:uint8 -> output:T, output_min:float, output_max:float, mkl_output:uint8, mkl_output_min:uint8, mkl_output_max:uint8; attr=T:type,allowed=[DT_QINT8, DT_QUINT8, DT_QINT32, DT_QINT16, DT_QUINT16]; attr=mode:string,default="SCALED",allowed=["MIN_COMBINED", "MIN_FIRST", "SCALED"]; attr=round_mode:string,default="HALF_TO_EVEN",allowed=["HALF_AWAY_FROM_ZERO", "HALF_TO_EVEN"]>; NodeDef: {{node import/resnet_model/conv2d_28/Conv2D_eightbit_quantize_resnet_model/Relu_24}}. (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
Aborted (core dumped)
I was unable to find documentation on how to quantize the official resnet model and run the fine tuning. Is there something I'm missing?
Thanks, Max
