Misc. bug: RISCV output bug when using rvv with vlen > 256bit

[grigohas]

Name and Version

./llama-cli latest version ubuntu linux riscv

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-cli

Problem description & steps to reproduce

The generated output of llama-cli in riscv env with rvv enabled consists of trash when vlen is greater than 256bits

First Bad Commit

No response

Relevant log output

No response

[JocelynPanPan]

Hello, I am a beginner and would like to ask what does the parameter "vlen" mean? I recently tested turning on RVV and the text generation result was the same as what you encountered. The results are as follows:

[grigohas]

Hello, I am a beginner and would like to ask what does the parameter "vlen" mean? I recently tested turning on RVV and the text generation result was the same as what you encountered. The results are as follows:

Hello, vlen is the vector register width. You can check this pull https://github.com/ggerganov/llama.cpp/pull/3453 where the developer told me that this might be a bug.

[JocelynPanPan]

OK, thanks for your reply!

[op21beyond]

Hello,

Have this issue resolved now? or do we have any workaround?

BTW, it seems that vlen > 256bits is not a necessary condition for this issue. In my RISCV ISA simulator and a silicon with vlen=128bits, the output tokens are broken.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.