ONE icon indicating copy to clipboard operation
ONE copied to clipboard

[Compiler FE] Convert dynamic batch to single batch

Open jinevening opened this issue 3 years ago • 2 comments

What

Let's implement an option to convert a model with a dynamic batch size to a model with a single batch.

For example,

onecc optimize --dynamic_batch_to_single_batch -i dynamic.circle -o single.circle

Why

The option will

  1. enable quantization of the model
  2. allow the model to be run on backends that do not support dynamic shapes

jinevening avatar Mar 29 '22 00:03 jinevening

dynamic batch size: where shape of input is like (-1, 224, 244, 3) where -1 is unknown(dynamic) batch size.

seanshpark avatar Mar 29 '22 02:03 seanshpark

More specifically, shape_signature is like (-1, 224, 224, 3). The below case is possible

shape: (1, 224, 224, 3) shape_signature: (-1, 224, 224, 3)

jinevening avatar Mar 29 '22 02:03 jinevening