Support batch-parallel execution of synchronously called trix models.

[ragmani]

[ragmani]

@nnfw-bot test tizen-gbs-builder

[ragmani]

I tested this draft in https://github.sec.samsung.net/RS7-RuntimeNTools/Self_Runner_Workflows/actions/runs/2069760

• Batch : 1

Package Filename /opt/workspace/2069760/mv1.q8
/opt/workspace/2069760/mv1.q8/result.tv2b.0 is generated.
===================================
MODEL_LOAD   takes 2.247 ms
PREPARE      takes 32.210 ms
EXECUTE      takes 8.658 ms
- MEAN     :  8.658 ms
- MAX      :  40.583 ms
- MIN      :  3.624 ms
- GEOMEAN  :  5.987 ms
===================================
Package Filename /opt/workspace/2069760/mv1.q16
/opt/workspace/2069760/mv1.q16/result.tv2b.0 is generated.
===================================
MODEL_LOAD   takes 1.364 ms
PREPARE      takes 95.705 ms
EXECUTE      takes 9.519 ms
- MEAN     :  9.519 ms
- MAX      :  12.131 ms
- MIN      :  8.316 ms
- GEOMEAN  :  9.452 ms
===================================

• Batch : 10

Batch size: 10
Package Filename /opt/workspace/2069760/mv1.q8
/opt/workspace/2069760/mv1.q8/result_parallel.tv2b.0 is generated.
===================================
MODEL_LOAD   takes 1.374 ms
PREPARE      takes 76.422 ms
EXECUTE      takes 30.272 ms
- MEAN     :  30.272 ms
- MAX      :  50.526 ms
- MIN      :  21.981 ms
- GEOMEAN  :  28.989 ms
===================================
Package Filename /opt/workspace/2069760/mv1.q16
/opt/workspace/2069760/mv1.q16/result_parallel.tv2b.0 is generated.
===================================
MODEL_LOAD   takes 1.392 ms
PREPARE      takes 154.804 ms
EXECUTE      takes 61.029 ms
- MEAN     :  61.029 ms
- MAX      :  80.850 ms
- MIN      :  52.740 ms
- GEOMEAN  :  60.403 ms
===================================

[ragmani]

@nnfw-bot test x64-builder

[ragmani]

@nnfw-bot test x64-builder cross-runtime-test-release

[hseok-oh]

Merged