[QNN] Support non-Android aarch64 Linux targets for QNN backend?

[tc-wolf]

I'm targeting Snapdragon 8 Gen 2 hardware, but it is using (non-Android) aarch64 Linux (under a QNX hypervisor) as the operating system. I've been able to use the QNN SDK directly to deploy models successfully using the GPU backend, but am wanting to use executorch with / on top of QNN.

In the example documentation I've read, the Android NDK is a dependency to use the QNN backend with Executorch. Would it be possible to allow for targeting non-Android aarch64 linux with Executorch + the QNN backend? I've been looking at altering some of the CMakeLists.txt to work around the Android requirement for a proof of concept, so some guidance or upstream changes would be greatly appreciated.

[chiwwang]

May I know if the below work for you? https://github.com/pytorch/executorch/issues/1527#issuecomment-1876428906

[tc-wolf]

I'll look into #1527 and see if that works, thanks!

[tc-wolf]

I've tried that workaround, but had to also comment out the code defining NATIVE_FP16 1 in half.h:

/*
#if defined(__GNUC__) || defined(__clang__)
#if defined(__aarch64__)
#ifndef __ARM_V8_ONLY__
#define NATIVE_FP16 1
#endif // __ARM_V8_ONLY__
#endif // __aarch64__
#endif // GNUC or clang
*/

otherwise, I get the following error:

[ 12%] Building CXX object CMakeFiles/executorch_no_prim_ops.dir/runtime/backend/interface.cpp.o
In file included from /home/ubuntu/executorch/../executorch/runtime/core/portable_type/complex.h:11,
                 from /home/ubuntu/executorch/../executorch/runtime/core/exec_aten/exec_aten.h:34,
                 from /home/ubuntu/executorch/../executorch/runtime/core/evalue.h:10,
                 from /home/ubuntu/executorch/../executorch/runtime/core/event_tracer.h:9,
                 from /home/ubuntu/executorch/../executorch/runtime/backend/backend_execution_context.h:11,
                 from /home/ubuntu/executorch/../executorch/runtime/backend/interface.h:13,
                 from /home/ubuntu/executorch/runtime/backend/interface.cpp:9:
/home/ubuntu/executorch/../executorch/runtime/core/portable_type/half.h:49:5: error: ‘_Float16’ does not name a type; did you mean ‘BFloat16’?
   49 |     _Float16 y;
      |     ^~~~~~~~
      |     BFloat16

I've experimented a bit with specifying the -mfp16-format=ieee as a compilation option or specifying the C++ / C versions, but I think that this is because _Float16 is a C type, but the header is being used by C++? I'm worried I have something with cross-compilation misconfigured, as I'm not experienced with cmake setup, and my environment is a bit ad-hoc.

I had to change the cmake toolchain file slightly, so I'm using is this:

set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR aarch64)
SET(CMAKE_HOST_SYSTEM_NAME Linux)
set(CMAKE_HOST_SYSTEM_PROCESSOR x86_64)
set(CMAKE_BUILD_WITH_INSTALL_RPATH TRUE)

set(CMAKE_C_COMPILER /usr/bin/aarch64-linux-gnu-gcc)
set(CMAKE_CXX_COMPILER /usr/bin/aarch64-linux-gnu-g++)

set(CMAKE_FIND_ROOT_PATH /usr/aarch64-linux-gnu/ /home/ubuntu/executorch/)

# adjust the default behavior of the FIND_XXX() commands:
# search programs in the host environment
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)

# search headers and libraries in the target environment
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY

Will test whether or not this works with this workaround for _Float16, but wanted to share progress so far.

[tc-wolf]

Slow but steady progress - ran into https://github.com/pytorch/executorch/issues/2955 (think that the instructions in setup.md have to be updated to add -DEXECUTORCH_BUILD_SDK=ON to the first build step), but after specifying that config option, the executor runner was able to compile.

I ran the model export script, and will need to test on my device to confirm works end-to-end.

Thanks for the information!

[tc-wolf]

Can confirm that can successfully export and run models with qnn-executor-runner, though the outputs I'm getting from the converted VIT model don't look close to the original's. I'll debug that some more on my own, but I think can close this, as I think that's most likely just an artifact of conversion or quantization.