Unexpected behavior in JIT mode

[erxiaozhou]

Build commands

I compile the code with commit id 40b430fd240ad45f5c7a7dd0239fa2425e7c294e. Platform: Ubuntu 20.04 CPU: amd64 compile:

export CC=/usr/lib/llvm-16/bin/clang
export CXX=/usr/lib/llvm-16/bin/clang++
cd product-mini/platforms/linux/;rm -rf build
cmake  -DWASM_ENABLE_AOT=1 -DWAMR_BUILD_JIT=1 -DWAMR_BUILD_FAST_JIT=0   -DCMAKE_INSTALL_PREFIX="xxx" -DWAMR_BUILD_REF_TYPES=1 -DCMAKE_BUILD_TYPE=Release  -DWASM_ENABLE_BULK_MEMORY=1 -DWAMR_BUILD_LIBC_WASI=0 -DWAMR_BUILD_LIBC_BUILTIN=1 -DWAMR_BUILD_SIMD=1 -Bbuild

execute:

iwasm --heap-size=0 --llvm-jit -f to_test <tc_name>

Cases

v128.load64_lane_2635_10_1703418883268537.zip

Actual behavior:

No exception

<0x0000000000000000 0xffffffffffffffff>:v128

Expected behavior:

An exception indicates that "0000523: error: unexpected opcode: 0xfd 0xff 0x1"

[TianlongLiang]

Hi, the reason you get both the error message fast jit compilation failed: Error: unsupported opcode and the successful execution result <0x0000000000000000 0xffffffffffffffff>:v128 is you are using the Multi-tier JIT enabled iwasm. By enabling the flag DWAMR_BUILD_FAST_JIT after your initial CMake cache variable setup, you would make both DWAMR_BUILD_FAST_JIT=1 and DWAMR_BUILD_JIT=1, making the iwasm Multi-tier JIT enabled.

Multi-tier JIT enabled iwasm will first generate fast JIT jitted code and use it and wait for LLVM jit jitted code to be ready and switch to use it. So the first error message displays the compilation failed for Fast JIT because it doesn't support SIMD opcodes now and then the second message comes from the successful running result of the LLVM JIT jitted code.

Here is a blog about multi-tier JIT, hope it helps. Feel free to reach out if you have more questions.

[erxiaozhou]

Hi, the reason you get both the error message fast jit compilation failed: Error: unsupported opcode and the successful execution result <0x0000000000000000 0xffffffffffffffff>:v128 is you are using the Multi-tier JIT enabled iwasm. By enabling the flag DWAMR_BUILD_FAST_JIT after your initial CMake cache variable setup, you would make both DWAMR_BUILD_FAST_JIT=1 and DWAMR_BUILD_JIT=1, making the iwasm Multi-tier JIT enabled.

Multi-tier JIT enabled iwasm will first generate fast JIT jitted code and use it and wait for LLVM jit jitted code to be ready and switch to use it. So the first error message displays the compilation failed for Fast JIT because it doesn't support SIMD opcodes now and then the second message comes from the successful running result of the LLVM JIT jitted code.

Here is a blog about multi-tier JIT, hope it helps. Feel free to reach out if you have more questions.

Thank you for your nice reply! The blog is useful! Meanwhile, I have two questions. First, according to the output of wasm-validate the attached test case is an invalid test case, which should be executed.

$ wasm-validate  v128.load64_lane_2635_10_1703418883268537.wasm

Output

0000523: error: unexpected opcode: 0xfd 0xff 0x1

Second, can I use the option DWAMR_BUILD_LAZY_JIT to build my iwasm? I think the option is interesting, but the blog does not introduce it.

Grateful and looking forward to your recovery!

[TianlongLiang]

For your first question, could you please tell me which version of wabt you are using? It may not have SIMD proposal enabled as the default.

For your second question, your build should have DWAMR_BUILD_LAZY_JIT=1 as default, it's not a JIT running mode like fast jit or llvm jit, but is a configuration for llvm jit.

From this document, I quote:

Note: By default, the LLVM Orc JIT with Lazy compilation is enabled to speedup the lanuching process and reduce the JIT compilation time by creating backend threads to compile the WASM functions parallely, and for the main thread, the functions in the module will not be compiled until they are firstly called and haven't been compiled by the compilation threads. If developer wants to disable the Lazy compilation, we can:

mkdir build && cd build
cmake .. -DWAMR_BUILD_JIT=1 -DWAMR_BUILD_LAZY_JIT=0
make

In which all the WASM functions will be previously compiled before main thread starts to run the wasm module.

[erxiaozhou]

I have tried multiple tools to execute the test case:

• wasmtime-cli 17.0.0 (37300d3f4 2023-12-20)
• wasm-validate (git~1.0.32-66-gfb31c1a2)
• wasmedge 0.13.5-73-ge41eb56b and they all reported the similar content indicating the test case is invalid.

Thank you for your nice reply!

[TianlongLiang]

Hi, my wabt is a little outdated, it's 1.0.29 and it displays another error message:

v128.load64_lane_2635_10_1703418883268537.wasm:0000523: error: type mismatch in f64x2.convert_low_i32x4_u, expected [v128] but got [... i32]
v128.load64_lane_2635_10_1703418883268537.wasm:00005a3: error: type mismatch in i64.ge_s, expected [i64, i64] but got [... v128, v128]
v128.load64_lane_2635_10_1703418883268537.wasm:00005ab: error: type mismatch at end of `if false` branch, expected [] but got [i64, i64

I tried to install the newest wabt but seems it has some kind of libc dynamic linking issues on Ubuntu 20, I will find a Ubuntu 22 environment and configure the same version of tools you have, then look into it. Will get back to you ASAP

[TianlongLiang]

Hi, after further investigation, we found out this test case is caused by the incorrect handling of the sub-opcode of the SIMD opcode, and #3115 should fix this problem. Thanks for spotting this bug for us