caffe quantization problem

I cant understand how did you find params for quantization such as scale_in scale_out scale_params. Is any automatic tool for this?

Mar 13 '18 11:03 apletea

Yes, please find the script at scripts/calibrator.py. The doc is available below: https://github.com/intel/caffe/wiki/Introduction-of-Accuracy-Calibration-Tool-for-8-Bit-Inference

Mar 13 '18 11:03 jgong5

After using new (generated) quantized prototxt file on deploy step error "F0314 13:33:43.486754 9157 mkldnn_batch_norm_layer.cpp:248] Check failed: BatchNormFwd_pd" are occured in compilation time. Could you something help with it ?

Mar 14 '18 11:03 DLegushev

May I know what model you are running? Batch normalization should be folded into the convolution in most cases. Are you running the model with TEST mode?

Mar 14 '18 11:03 jgong5

SqueezeNet model. But in this case batch normalization is going as separated layer. https://drive.google.com/open?id=1O8zb-y5LJysAaFXgFuH-lieO6BB4IwH4

Mar 14 '18 12:03 DLegushev

How did you run the model? If you are using "caffe time", please add "-phase TEST" and "-forward_only" to the command line.

Mar 14 '18 12:03 jgong5

I used both "caffe time" with this parameters and just "caffe test". Also i try used this quantized model for deploy aims. The error is the same.

Mar 15 '18 08:03 DLegushev

It solves by adding in the deploy.prototxt engine : "CAFFE"

Mar 20 '18 15:03 apletea

@apletea But quantization is basically disabled with "CAFFE" engine.

Mar 21 '18 00:03 jgong5

@jgong5 so its means that we still have a problem) By the way adding "CAFFE" engine speed up CPU forward from 0.05 to 0.03

Mar 21 '18 09:03 apletea

Does the int8 optimization support AVX2 on Haswell?

Mar 22 '18 07:03 fei1107

@fei1107 No, only Skylake server supports int8 now.

Mar 22 '18 07:03 jgong5

Hi @DLegushev, would u please share us the corresponding caffemodel file？ So we can reproduce your result/issue locally.

Mar 22 '18 07:03 guomingz

Yes, of course. https://drive.google.com/file/d/1P9a1P7gefRqsK7tETy3NEDQfKAL_0tKX/view?usp=sharing

Mar 22 '18 08:03 DLegushev

Hi @jgong5 , int8 inference only work with AVX512 now?

Dec 13 '18 08:12 TianzhongSong