copilot-cli icon indicating copy to clipboard operation
copilot-cli copied to clipboard
verified publisher icon github

Handling of unicode characters in the CLI integration in VS Code

Open DonJayamanne opened this issue 1 month ago • 3 comments

Describe the bug

Transferring issue to this repo as VS Code doesn't handle execution of terminal commands, the SDK does this. See another issue from another user https://github.com/microsoft/vscode/issues/275721#issuecomment-3585563373

Type: Bug

The screenshot shows it. The model produced a cat command invocation to write a file. The content contains unicode characters. The outcome is scrambled. This is probably a model issue, but we should first verify.

Image

Extension version: 0.33.2025110501 VS Code version: Code - Insiders 1.106.0-insider (1a756d609ca6220dbd315a779b9ef1baf8a761f2, 2025-11-05T05:02:46.325Z) OS version: Darwin arm64 24.6.0 Modes:

Prompt Details 
Debug: [streamChoices] choice { index: 0, finish_reason: null }
Debug: [streamChoices] chunk data: {"choices":[{"text":"\n","index":0,"finish_reason":"stop"}]}

Debug: [streamChoices] choice { text: '\n', index: 0, finish_reason: 'stop' }
Debug: [streamChoices] completion.finishReason n {
properties: {
headerRequestId: '4a8f1e23-3eaa-4870-93e6-6f88227e7b99',
opportunityId: 'icr-49db8784-a583-45d1-b01e-0d679d4e8585',
copilot_build: '1',
copilot_buildType: 'nightly',
languageId: 'jsonc',
afterAccept: 'false',
isSpeculative: 'false',
gitRepoInformation: 'unavailable',
engineName: '',
engineChoiceSource: 'default',
isMultiline: 'true',
isCycling: 'false',
endpoint: 'completions',
uiKind: 'ghostText',
temperature: '0',
n: '1',
stop: 'unset',
logit_bias: 'null',
completionChoiceFinishReason: 'stop'
},
measurements: { documentLength: 26903, documentLineCount: 841 },
issuedTime: 66493998.814416,
filtersAndExp: {
filters: hj { filters: [Object] },
exp: n { variables: {}, features: '' }
},
displayedTime: undefined
}
Info: [streamChoices] solution 0 returned. finish reason: [stop]
Debug: [streamChoices] solution 0 details: finishOffset: [undefined]
Debug: [streamChoices] chunk data: [DONE]


Debug: [streamChoices] request done: headerRequestId: [4a8f1e23-3eaa-4870-93e6-6f88227e7b99] model deployment ID: []
Debug: [streamChoices] request stats: ket {
choices: Map(1) { 0 => Tet { yieldedTokens: 2, seenTokens: 2 } }
}
Debug: [ghostText] All choices redacted
Debug: [AsyncCompletionManager] [4a8f1e23-3eaa-4870-93e6-6f88227e7b99] Request failed with all choices redacted
Debug: [AsyncCompletionManager] [4a8f1e23-3eaa-4870-93e6-6f88227e7b99] No matching completions found
Debug: [getCompletions] No ghostText produced -- empty: received no results from async completions
Debug: [context keys] Window state change. Needs offline check: false, active: true, focused: false.
Debug: [context keys] Window state change. Needs offline check: false, active: false, focused: false.
Debug: [context keys] Window state change. Needs offline check: false, active: false, focused: true.
Trace: No test results
Debug: [context keys] Window state change. Needs offline check: false, active: true, focused: true.
Debug: [context keys] Window state change. Needs offline check: false, active: true, focused: false.
Debug: [context keys] Window state change. Needs offline check: false, active: true, focused: true.
Trace: [GitServiceImpl][doOpenRepository] Active repository: {"_repo":{"rootUri":{"$mid":1,"external":"file:///Users/kmaetzel/Development/projects/plan-tooling","path":"/Users/kmaetzel/Development/projects/plan-tooling","scheme":"file"},"inputBox":{},"state":{},"ui":{}},"rootUri":{"$mid":1,"external":"file:///Users/kmaetzel/Development/projects/plan-tooling","path":"/Users/kmaetzel/Development/projects/plan-tooling","scheme":"file"},"headBranchName":"satisfied-buzzard","headCommitHash":"1d3fb0e2f92a58b597a2ee72e16a448f14a35781","upstreamBranchName":"satisfied-buzzard","upstreamRemote":"origin","isRebasing":true,"remotes":["origin"],"remoteFetchUrls":["[email protected]:kieferrm/plan-tooling.git"],"changes":{"mergeChanges":[],"indexChanges":[],"workingTree":[],"untrackedChanges":[]},"_onDidChangeSignal":{"_observers":{},"debugName":"Observable Signal From Event"},"headBranchNameObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"headCommitHashObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"upstreamBranchNameObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"upstreamRemoteObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"isRebasingObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"_checkIsIgnored":{"_waitingTimeMs":1000,"_queue":[],"_timeout":null},"_isIgnored":{"_map":{},"_map2":{}}}
Trace: No test results
Trace: [GitServiceImpl][doOpenRepository] Active repository: {"_repo":{"rootUri":{"$mid":1,"external":"file:///Users/kmaetzel/Development/projects/plan-tooling","path":"/Users/kmaetzel/Development/projects/plan-tooling","scheme":"file"},"inputBox":{},"state":{},"ui":{}},"rootUri":{"$mid":1,"external":"file:///Users/kmaetzel/Development/projects/plan-tooling","path":"/Users/kmaetzel/Development/projects/plan-tooling","scheme":"file"},"headBranchName":"satisfied-buzzard","headCommitHash":"1d3fb0e2f92a58b597a2ee72e16a448f14a35781","upstreamBranchName":"satisfied-buzzard","upstreamRemote":"origin","isRebasing":true,"remotes":["origin"],"remoteFetchUrls":["[email protected]:kieferrm/plan-tooling.git"],"changes":{"mergeChanges":[],"indexChanges":[],"workingTree":[],"untrackedChanges":[]},"_onDidChangeSignal":{"_observers":{},"debugName":"Observable Signal From Event"},"headBranchNameObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"headCommitHashObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"upstreamBranchNameObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"upstreamRemoteObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"isRebasingObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"_checkIsIgnored":{"_waitingTimeMs":1000,"_queue":[],"_timeout":null},"_isIgnored":{"_map":{},"_map2":{}}}
Debug: Loaded 6 events for session e7a686aa-b17b-4ead-97ad-01da6ce6d3fa
Debug: Loaded 40 events for session 5d4e72b7-89bf-4d14-bfce-27e07f953491
Debug: Loaded 6 events for session 4487feb9-a4fb-415f-ae4f-0cb371be9e93
Debug: Loaded 6 events for session f0f8969c-293a-401c-95c3-aef24a210d6e
Debug: Loaded 21 events for session 7077e290-0907-4b08-a1a7-8fb3939a34f4
Debug: Loaded 58 events for session 751be174-3aaa-4f05-8846-7079256cb9e0
Debug: Loaded 34 events for session df3f600d-8a28-43ac-8e92-2fbd7af43288
Debug: Loaded 55 events for session b7d5b6af-5318-4751-8c9e-f04df599e34b
Debug: Loaded 5 events for session 67b3c130-9399-4463-b777-87269dd8080d
Debug: Loaded 6 events for session d06c5f97-fc17-47b2-91d3-7432e1f56aad
Debug: Loaded 42 events for session 5c334910-5c84-46fb-86fa-b4b3aeeff088
Debug: Found 11 JSONL sessions and 2 legacy sessions
Info: Loaded legacy session 5f2e84f7-f1bf-448c-8feb-291623105e93 from session_5f2e84f7-f1bf-448c-8feb-291623105e93_1760028210888.json
Info: Loaded legacy session 4ccbd271-2bb5-4c7e-8de2-d35781b76be4 from session_4ccbd271-2bb5-4c7e-8de2-d35781b76be4_1759785249774.json
Trace: [GitServiceImpl][doOpenRepository] Active repository: {"_repo":{"rootUri":{"$mid":1,"external":"file:///Users/kmaetzel/Development/projects/plan-tooling","path":"/Users/kmaetzel/Development/projects/plan-tooling","scheme":"file"},"inputBox":{},"state":{},"ui":{}},"rootUri":{"$mid":1,"external":"file:///Users/kmaetzel/Development/projects/plan-tooling","path":"/Users/kmaetzel/Development/projects/plan-tooling","scheme":"file"},"headBranchName":"satisfied-buzzard","headCommitHash":"1d3fb0e2f92a58b597a2ee72e16a448f14a35781","upstreamBranchName":"satisfied-buzzard","upstreamRemote":"origin","isRebasing":true,"remotes":["origin"],"remoteFetchUrls":["[email protected]:kieferrm/plan-tooling.git"],"changes":{"mergeChanges":[],"indexChanges":[],"workingTree":[],"untrackedChanges":[]},"_onDidChangeSignal":{"_observers":{},"debugName":"Observable Signal From Event"},"headBranchNameObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"headCommitHashObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"upstreamBranchNameObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"upstreamRemoteObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"isRebasingObs":{"_observers":{},"_debugNameData":{},"_state":0,"_updateCount":0,"_dependencies":{},"_dependenciesToBeRemoved":{},"_isUpdating":false,"_isComputing":false,"_didReportChange":false,"_isInBeforeUpdate":false,"_isReaderValid":false,"_removedObserverToCallEndUpdateOn":null},"_checkIsIgnored":{"_waitingTimeMs":1000,"_queue":[],"_timeout":null},"_isIgnored":{"_map":{},"_map2":{}}}
Trace: [GitServiceImpl][getRepositoryFetchUrls] URI: file:///Users/kmaetzel/Development/projects/plan-tooling/cli/integration-tests/validate-plan-test.ts
Trace: [GitServiceImpl][getRepositoryFetchUrls] URI: file:///Users/kmaetzel/Development/projects/plan-tooling/cli/integration-tests/validate-plan-test.ts
Trace: [GitServiceImpl][getRepositoryFetchUrls] Remotes (open repository): {"rootUri":{"$mid":1,"path":"/Users/kmaetzel/Development/projects/plan-tooling","scheme":"file"},"remoteFetchUrls":["[email protected]:kieferrm/plan-tooling.git"]}
Trace: [GitServiceImpl][getRepositoryFetchUrls] Remotes (open repository): {"rootUri":{"$mid":1,"path":"/Users/kmaetzel/Development/projects/plan-tooling","scheme":"file"},"remoteFetchUrls":["[email protected]:kieferrm/plan-tooling.git"]}
Trace: No test results
Trace: RemoteEmbeddingsComputer::computeEmbeddings started
Trace: RemoteEmbeddingsComputer::computeEmbeddings success. Elapsed 297.76320800185204
Debug: Related Information: Remote similarly request took 298ms
Debug: Related Information: Successfully Calculated, returning 302ms
Trace: No test results
Debug: [context keys] Window state change. Needs offline check: false, active: true, focused: false.
Debug: [context keys] Window state change. Needs offline check: false, active: true, focused: true.
Request IDs 



System Info





Item
Value






CPUs
Apple M4 (10 x 2400)




GPU Status
2d_canvas: enabled
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: enabled_on
trees_in_viz: disabled_off
video_decode: enabled
video_encode: enabled
webgl: enabled
webgl2: enabled
webgpu: enabled
webnn: disabled_off		




Load (avg)
2, 2, 2




Memory (System)
32.00GB (1.12GB free)




Process Argv
--enable-proposed-api lszomoru.lszomoru-proposed-api-sample --enable-proposed-api donjayamanne.kusto --crash-reporter-id ef10dfea-3602-4c74-ba1a-109ce7da7825




Screen Reader
no




VM
0%







A/B Experiments

vsliv368cf:30146710
pythonvspyt551cf:31249598
nativeloc1:31118317
dwcopilot:31158714
dwoutputs:31242946
copilot_t_ci:31333650
g012b348:31231168
6gi0g917:31259950
pythonrdcb7:31268811
pythonpcpt1:31399616
6518g693:31302842
6abeh943:31336334
envsactivate1:31349248
editstats-enabled:31346256
aa_c:31379597
c12345:31413568
cloudbuttont:31366566
aihoversummaries_t:31413033
todos-1:31405333
qwen_all_req:31396049
3efgi100_wstrepl:31403338
trigger-command-fix:31379601
chat-sessions-view-t:31407655
auto_model_enabled:31396818
use-responses-api:31390341
applesignin:31399434
9i271672:31411680
ddidt:31398484
je187915:31401257
control_gfa0i828:31401959
f1721337:31405248
afag3459:31405336
disable_auto_fix_gb0j8799:31411651
agentic_ask_h7d75428:31406124
ec5jj548:31408805
copilot-nes-oct-t:31413017
metis_rerank_treatment1:31411021
ff8f5884:31411053
terminalsuggestenabled:31411718
cmp-ext-treat:31412886
none:31413573







Affected version


No response


Steps to reproduce the behavior


No response


Expected behavior


No response


Additional context


No response


                                    

                                    

                                        

                                            DonJayamanne avatar
                                            
                                                Nov 28
                                                '25  07:11
                                            
                                            
                                                DonJayamanne
                                            
                                        

                                        

                                    

                                    

                                

                                                                    

                                    

                                        
Also reported by doggy8088
2 days ago


I have similar situation. When LLM generated non-English characters in the PowerShell. The PowerShell will not able to read/execute.

Image

                                    

                                    

                                        

                                            DonJayamanne avatar
                                            
                                                Nov 28
                                                '25  07:11
                                            
                                            
                                                DonJayamanne
                                            
                                        

                                    

                                    

                                

                                                                    

                                    

                                        
Originally reported here https://github.com/microsoft/vscode/issues/275721#issuecomment-3585555009


                                    

                                    

                                        

                                            DonJayamanne avatar
                                            
                                                Nov 28
                                                '25  08:11
                                            
                                            
                                                DonJayamanne
                                            
                                        

                                    

                                    

                                

                                                                    

                                    

                                        
@devm33 /cc


                                    

                                    

                                        

                                            DonJayamanne avatar
                                            
                                                Nov 28
                                                '25  08:11
                                            
                                            
                                                DonJayamanne