microsoft
Enhance constant folding for Shape node
Description: Enhance constant folding for Shape node.
In ConstantFolding optimizer, if a Shape node's input shape have concrete values in all dimensions, then we replace this Shape node with an initializer filled with the concrete 1-D shape values.
In some 1P model, we see many subgraph related to shape operation. The Shape node's input shape have some dimension being symbolic value, the others being concrete values. Those Shape nodes usually are followed by Slice or Gather node.
If the dimension Slice/Gather nodes want to retrieve is concrete values, we can also constant fold this Slice/Gather node.
Here is the op count comparison without and with this PR:
Unsqueeze: 982-->252 Slice: 652->120 Squeeze: 628->2 Gather: 563->367 ConcatTraining: 483->195 Clip: 96 -> 0
Motivation and Context
- Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here.
In the models which I have seen, usually the Gather\Slice nodes retrieve the symbolic dimension. For cases where a known dim is being retrieved, wondering if the converters can do this optimization on their end. It is good to have this in our runtime but if the converters can also add a similar optimization, we will see more simple/efficient ONNX models.
Will create a new specialized shape optimizer for this, to avoid any backward incompatibility.