Batch Image Inference using PyArmNN Error

[AliSheheryar]

I am trying to have a batch inference on my image list( 2 images) using Pyarmnn on Arm-v7l device. Single Image inference is doing fine. make_input_tensor function produces the required tensors but when fed to enqueue_workload function, it throws a runtime error.

File "input_tensor_testing.py", line 172, in main runtime.EnqueueWorkload(net_id, input_tensors, output_tensors) File "/usr/lib/python3/dist-packages/pyarmnn/_generated/pyarmnn.py", line 4118, in EnqueueWorkload return _pyarmnn.IRuntime_EnqueueWorkload(self, networkId, inputTensors, outputTensors) RuntimeError: Number of inputs provided does not match network.

I am using Tflite file for Yolo_v3 for object detection.

[MikeJKelly]

Hi @AliSheheryar

how are you passing the two images? Are you passing them one image at a time or both at once? If you want to perform inferences on two images you will have to call enqueue_workload twice, once with each image.

Best regards, Mike

[AliSheheryar]

Mike thanks for your help. I want to run inference on two images at the same time. enqueue_workload function can also work on list on input images a/c to ARMNN package documentation.A little explanantion would be helpful. Furthermore, on CPUAcc as backend , what is the maximum FPS you can achieve using CPU only? I am getting 1FPS and wondering if there is another way to enhance FPS. GPU is not an option as ARMv7 devices don't have NVIDIA GPU power.

[FrancisMurtagh-arm]

Hi @AliSheheryar,

We would recommend upgrading to an ARMv8 device as most of our optimizations such as those related to NEON are not supported on ARMv7.

Could you provide a code snippet of what you are trying to do re: "to run inference on two images at the same time".

Thanks, Francis.

[AliSheheryar]

Thank you for you cooperation. I was able to run the armv7 with 1Frame/dev speed with Neon acceleration module but no GpU. I can provide the whole code and stuff. Open source is the way to move forward.

On Fri, Jan 6, 2023 at 7:02 PM FrancisMurtagh-arm @.***> wrote:

Hi @AliSheheryar https://github.com/AliSheheryar,

We would recommend upgrading to an ARMv8 device as most of our optimizations such as those related to NEON are not supported on ARMv7.

Could you provide a code snippet of what you are trying to do re: "to run inference on two images at the same time".

Thanks, Francis.

— Reply to this email directly, view it on GitHub https://github.com/ARM-software/armnn/issues/655#issuecomment-1373685188, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANO5JNONF3KUULDFZPBICE3WRAQW5ANCNFSM5ZIFJNSQ . You are receiving this because you were mentioned.Message ID: @.***>

[AliSheheryar]

Not a batch of images. Just one image per function call. Sorry for not answering earlier about the actual question.

On Sat, Jan 7, 2023 at 6:04 PM Ali Sheheryar @.***> wrote:

Thank you for you cooperation. I was able to run the armv7 with 1Frame/dev speed with Neon acceleration module but no GpU. I can provide the whole code and stuff. Open source is the way to move forward.

On Fri, Jan 6, 2023 at 7:02 PM FrancisMurtagh-arm < @.***> wrote:

Hi @AliSheheryar https://github.com/AliSheheryar,

We would recommend upgrading to an ARMv8 device as most of our optimizations such as those related to NEON are not supported on ARMv7.

Could you provide a code snippet of what you are trying to do re: "to run inference on two images at the same time".

Thanks, Francis.

— Reply to this email directly, view it on GitHub https://github.com/ARM-software/armnn/issues/655#issuecomment-1373685188, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANO5JNONF3KUULDFZPBICE3WRAQW5ANCNFSM5ZIFJNSQ . You are receiving this because you were mentioned.Message ID: @.***>

[matthewsloyanARM]

Hi @AliSheheryar,

Thanks for getting in touch. If this is still an issue, would you be able to provide a code snippet here and I can try and reproduce it for you?

Kind regards,

Matthew

[matthewsloyanARM]

Hi @AliSheheryar,

I am going to close this issue as I haven't heard back in a little while, however if you are still having the issues feel free to reopen the ticket. Thanks again for getting in touch.

Kind regards,

Matthew