dummy config float

[mbelda]

Hello,

I am trying to generate a 16x16 FP32 Gemmini, but because it ocuppies so much luts I am going with the dummy config option (as you said me some time ago on the Chipyard google group).

Right now, I have been able to generate the 16x16 Int Gemmini with input-size 32b to match floats, but when I try to run the inference of a cnn a free() pointer error is thrown.

Since the error is not descriptive at all I don't know what could be happening. Any ideas?

[hngenc]

Pointer errors are usually caused by gemmini_params.h files which don't match each other.

Can you make sure that your systolic_params_fp32.h file matches the gemmini_params.h which was created by your dummy config?

After that, make sure to rm -rf ./build/ and recompile ONNX-Runtime.

[mbelda]

I am a bit confused with this particular case. I have created a dummy 32bit int gemmini config, so my gemmini params are on the file systolic_params_int8.h. Is that right?

Also, I have onnx built for fp32, do I need to rebuild onnx for this case? But with which options? I usually use this options for float onnxruntime_SYSTOLIC_INT8=ON onnxruntime_SYSTOLIC_FP32= OFF but there is no option for int32.

So, how do I proceed?

[hngenc]

1. While we don't explicitly support int32 in ONNX-Runtime, I believe that if you just replace systolic_params_int8.h with the int32 gemmini_params.h and rebuild, then it should work. I haven't tested it myself, but I've skimmed the code, and I think it should work fine. (If you're curious why, it's because gemmini_params.h has a typedef called elem_t, and ONNX Runtime [uses that typedef](https://github.com/ucb-bar/onnxruntime-riscv/blob/0c8c9b4f881b5f31d32c6b5a76cac4ee14a8f338/onnxruntime/core/mlas/lib/systolic/systolic.cpp#L720 rather than using int8 types directly).
2. I think you would still build with onnxruntime_SYSTOLIC_INT8=ON onnxruntime_SYSTOLIC_FP32=OFF, even for int32.

We haven't tested this ourselves though, so I'm curious to see if this works.