If input contains specific content when testing a single block, the test do not work

[cowcomic]

Self Checks

• [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.8.2

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

create a block test input as "=1 group by" the pattern of the string as "={num}{space}group{space}by{space}" start run it seems to be running, but not a really running. The result is still the result of the last valid input

✔️ Expected Behavior

The result is still the result of the last valid input

❌ Actual Behavior

No response

[dosubot[bot]]

The issue you're encountering with the block test not working when the input contains specific content like =1 group by is likely related to how the input is tokenized and classified. The tokenizer in the pgsql.js file is designed to handle such inputs by classifying components as operators, numbers, and keywords. Specifically, = is classified as an operator, 1 as a number, and group and by as keywords ^[1].

However, the behavior you're describing, where the test seems to run but returns the result of the last valid input, suggests there might be a caching mechanism or an issue with how the test results are updated. According to the documentation, Dify updates the test results when you enter common user questions in the Source Text input box and click Test. There is no mention of conditions under which Dify might reuse previous results ^[2].

To address this issue, you might want to check if there are any caching mechanisms in place that could be affecting the test results. Additionally, ensure that the input is correctly tokenized and classified according to the defined rules in the pgsql.js file. If the problem persists, it might be worth investigating further into the specific implementation details of the block test mechanism and any potential caching behavior.

To continue talking to Dosu, mention @dosu.

[Wu-Jiayang]

You can try this: https://github.com/langgenius/dify/pull/7955

[Wu-Jiayang]

This can solve your problem

[cowcomic]

This can solve your problem

any plan for merge it to a version?

[Wu-Jiayang]

This can solve your problem

any plan for merge it to a version?

I opened a PR, but I'm not sure when they will merge it.

[Wu-Jiayang]

Maybe they will merge it only if we give them a push.

[dosubot[bot]]

Hi, @cowcomic. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary:

• You reported a bug in Dify version 0.8.2 where specific input patterns cause incorrect test outputs.
• I suggested the issue might be due to input tokenization and caching mechanisms.
• @Wu-Jiayang proposed a solution with PR #7955 but mentioned uncertainty about the merge timeline.
• You inquired about when the fix would be merged into a release version.

Next Steps:

• Please let us know if this issue is still relevant to the latest version of Dify. If so, you can keep the discussion open by commenting here.
• If there is no further activity, this issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!