[Quantization] Precision loss graph instrumentation

[mciprian13]

In terms of quantization it would be very helpful to understand where exactly in the graph (what nodes) does most of the quantization error (precision loss) occur. Since the accuracy of the quantization drops dramatically for large/deep models especially when using more rough quantization schemas (e.g. symmetric or symmetric_with_power2_scale) it would be very helpful to have a graph instrumentation mode which:

• when quantizing the graph also keeps the floating point version in parallel
• computes for every node the output error (some metric) between the quantized and floating-point version
• dumps the results in a format which can be over-imposed on the model such that a heat (color) map can be drawn to depict which layers are most responsible with the precision loss

[mciprian13]

@jfix71 WDYT about this? I really don't have time for this (and most probably you don't have also) but if this idea sounds reasonable/doable/nice-to-have then maybe we can work together on this in small increments.

[jfix71]

@mciprian13 Sounds cool! As you expected I don't have time time to work on this right now. As inspiration, it sounds somewhat similar to how our quantization profiling works -- adding a meta node to every node that observes the values flowing through the graph. So hopefully it wouldn't be too hard to implement in a similar fashion. Heatmap wouldn't be too bad either, to simply dump the dotty graph with specific colors for each Node.