Chat return content is too slow, a question to wait 8s to have a result, is not as fast as chatgpt response speed

[Aoyia]

[albertkun]

I would like to add that chat return's super slow response only started occurring after the latest VS Code update (1.87) and prior builds had much faster response time. (Slow as in like... 1-2 words per second... which is bad...)

I am using the inline chat (not the Co-pilot chat tab on the launcher). This might be related to the fix to the chat that stopped users from pressing "up" or "down" which would empty the chat?

Or perhaps this new context functionality:

No idea.. but it is unbearably slow now.

[Migxa]

In my case the chat response generation is so slow that it times out on its own failing to give response or stops its own response mid sentence.

However, the code suggestion in the code editor itself seem to work with the same speed as before.

[digitarald]

@albertkun @Aoyia sorry to hear, we haven't seen any slowdown in other reports. Can you please provide the output from Copilot Chat logs using VS Code's output pane?

[albertkun]

thanks for looking into this, but i don't think there's anything abnormal in the logs:

2024-03-26 22:46:15.721 [info] [code-referencing] Public code references are enabled.
2024-03-26 23:11:16.464 [info] [fetcher] Using Helix fetcher.
2024-03-26 23:11:16.464 [info] [code-referencing] Public code references are enabled.
2024-03-26 23:36:17.035 [info] [fetcher] Using Helix fetcher.
2024-03-26 23:36:17.035 [info] [code-referencing] Public code references are enabled.
2024-03-27 19:06:51.461 [info] [fetcher] Using Helix fetcher.
2024-03-27 19:06:51.462 [info] [code-referencing] Public code references are enabled.

i am thinking that it could be parsing of response issue? not too sure, but it seems like the issue mainly happens for after i've had an in-line chat session that is long. i've been just restarting vs code when it gets to be too slow.. which isnt an ideal workaround..

[jimthompson5802]

@digitarald I'm experiencing the same slow Copilot Chat response.

Copilot Chat Prompt

example code to do linear regression

Output from Copilot Chat Log from two different time periods

2024-04-12 14:47:49.269 [info] [chat fetch] url https://api.githubcopilot.com/chat/completions
2024-04-12 14:47:49.270 [info] [chat fetch] modelMaxPromptTokens 3072
2024-04-12 14:47:49.271 [info] [chat fetch] modelMaxResponseTokens 3709
2024-04-12 14:47:49.271 [info] [chat fetch] chat model gpt-4
2024-04-12 14:48:05.185 [info] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 15912 ms
2024-04-12 14:48:05.368 [info] [streamMessages] message 0 returned. finish reason: [stop]
2024-04-12 14:48:05.371 [info] [streamChoices] request done: requestId: [18b52ee9-78e7-4af6-b9d7-ad9ced72b819] responseId: [18b52ee9-78e7-4af6-b9d7-ad9ced72b819] model deployment ID: []
2024-04-12 14:48:09.502 [info] [chat fetch] url https://api.githubcopilot.com/chat/completions
2024-04-12 14:48:09.503 [info] [chat fetch] modelMaxPromptTokens 7168
2024-04-12 14:48:09.503 [info] [chat fetch] modelMaxResponseTokens 4096
2024-04-12 14:48:09.503 [info] [chat fetch] chat model gpt-3.5-turbo
2024-04-12 14:48:10.242 [info] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 736 ms
2024-04-12 14:48:10.247 [info] [streamMessages] message 0 returned. finish reason: [stop]
2024-04-12 14:48:10.249 [info] [streamChoices] request done: requestId: [fdfc8b23-c3f2-46b5-b2da-67f7eec90cec] responseId: [fdfc8b23-c3f2-46b5-b2da-67f7eec90cec] model deployment ID: [x4dff5e5d11fc]



2024-04-13 07:10:26.590 [info] [chat fetch] url https://api.githubcopilot.com/chat/completions
2024-04-13 07:10:26.591 [info] [chat fetch] modelMaxPromptTokens 3072
2024-04-13 07:10:26.591 [info] [chat fetch] modelMaxResponseTokens 3709
2024-04-13 07:10:26.592 [info] [chat fetch] chat model gpt-4
2024-04-13 07:10:34.405 [info] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 7812 ms
2024-04-13 07:10:34.482 [info] [streamMessages] message 0 returned. finish reason: [stop]
2024-04-13 07:10:34.483 [info] [streamChoices] request done: requestId: [e96236d7-8896-41d1-bf66-9d6e6a96adbe] responseId: [e96236d7-8896-41d1-bf66-9d6e6a96adbe] model deployment ID: []
2024-04-13 07:10:36.226 [info] [chat fetch] url https://api.githubcopilot.com/chat/completions
2024-04-13 07:10:36.227 [info] [chat fetch] modelMaxPromptTokens 7168
2024-04-13 07:10:36.227 [info] [chat fetch] modelMaxResponseTokens 4096
2024-04-13 07:10:36.227 [info] [chat fetch] chat model gpt-3.5-turbo
2024-04-13 07:10:36.829 [info] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 601 ms
2024-04-13 07:10:36.834 [info] [streamMessages] message 0 returned. finish reason: [stop]
2024-04-13 07:10:36.835 [info] [streamChoices] request done: requestId: [70f20b2a-13c3-4103-96b5-956c737d3be6] responseId: [70f20b2a-13c3-4103-96b5-956c737d3be6] model deployment ID: [x4dff5e5d11fc]

As I understand the log output, it appears that it took about 15.912 seconds to get a response in the first time period and 7.812 seconds in the second time period. If I try the same prompt in ChatGPT (GPT4), the time to see the first lines of recommended code is couple of seconds or less.

Here are the particulars for my setup:

Desktop

Win10 Virtual Desktop
VCPU: Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz   2.30 GHz  (2 processors)
Installed RAM: 24GB

Visual Studio Code

Version: 1.87.2 (system setup)
Commit: 863d2581ecda6849923a2118d93a088b0745d9d6
Date: 2024-03-08T15:20:17.278Z
Electron: 27.3.2
ElectronBuildId: 26836302
Chromium: 118.0.5993.159
Node.js: 18.17.1
V8: 11.8.172.18-electron.0
OS: Windows_NT x64 10.0.19045

Installed extensions

[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]

Visual Studio Code Settings

{
    "terminal.integrated.defaultProfile.windows": "Command Prompt",
    "editor.fontSize": 16,
    // "github.copilot.enable": {
    //     "*": true,
    //     "plaintext": false,
    //     "markdown": false,
    //     "scminput": false
    // },
    "github.copilot.advanced": {

        "debug.useNodeFetcher": true
    }
    // "http.proxyStrictSSL": false
}

[vs-code-engineering[bot]]

Hey @digitarald, this issue might need further attention.

@Aoyia, you can help us out by closing this issue if the problem no longer exists, or adding more information.

[digitarald]

@alexdima with the latest fetcher work, should we close it?

[alexdima]

I will leave this up to @chrmarti .

[chrmarti]

I'm not aware of recent reports like this and assume that this would have been a shortage of capacity or partial failure on the server side that has since been fixed.

Closing for now.