[BUG] In multi - round conversations, the replies generated by the Phi3 Medium model become chaotic

[mfjiang]

Describe the bug In multi - round conversations, the replies generated by the Phi3 Medium model become chaotic

To Reproduce Steps to reproduce the behavior:

1. Go to 'samples'
2. Click on 'Chat'
3. Chat with Phi 3 Medium GPU
4. See error

Expected behavior Provide natural reply content.

Screenshots

Please complete the following information:

Additional context

[zateutsch]

@mfjiang Would you mind sharing what device you are on?

Also, does the same behavior occur when running the CPU version of the model?

[mfjiang]

@mfjiang Would you mind sharing what device you are on?

Also, does the same behavior occur when running the CPU version of the model?

My device is RTX3060 12GB, and it runs AI models normally in the WSL2 environment.

I haven't tried inferring CPU version of themodel yet. I will give it a try later.

[nmetulev]

Hi @mfjiang, have you been able to reproduce this issue with non gpu variant of the model?

[mfjiang]

Hi @mfjiang, have you been able to reproduce this issue with non gpu variant of the model?

I tried the CPU version of phi-3 mini, and it seems to perform a bit better, but it's still not ideal. In the screenshot, the second response shows signs of hallucination(4. Vue.js：Vue.js 是一个开源的框架，用于构建用户界面。它采用懵懵的概念), and the response is incomplete.

my PC info: CPU 12th Gen Intel(R) Core(TM) i5-12400F 2.50 GHz RAM 32.0 GB (31.8 GB 可用)