onnxruntime icon indicating copy to clipboard operation
onnxruntime copied to clipboard
verified publisher icon microsoft

Share scalar constant initializer

Open pengwa opened this issue 3 years ago • 0 comments

Description: Share scalar constant for same data type, value and shape.

Share initializer for those who hold same value in same type and shape, currently only handle scalar value or 1-D single value array.

The transformation itself did not bring much impact on memory/perf, instead is helpful to simplify the graph, making it easier for common subexpression eliminations (CSE). Imagine graphs like this:

image

Add is NOT shared as inputs of Clip after CSE transformation because, all Add's second constant input are different NodeArg*, so if we change all constant initializer share the same NodeArg*, then only one Add will be preserved after CSE transformation. There are few other similar cases in one of 1P deberta models.

E2E measurement on 1P DEBERTA model, we see an increase from SamplesPerSec=562.041593991271 to 568.0106130440271, 1.07% gains.

Motivation and Context

  • Why is this change required? What problem does it solve?
  • If it fixes an open issue, please link to the issue here.

pengwa avatar Sep 07 '22 13:09 pengwa