opencode icon indicating copy to clipboard operation
opencode copied to clipboard

feat(opencode): add copilot specific provider to properly handle copilot reasoning tokens

Open SteffenDE opened this issue 5 days ago • 9 comments

What does this PR do?

This PR adds a copilot specific provider for the copilot completions API (there's already copilot specific code for the responses API). The code already states that this code is only meant for copilot, so I decided to rename the folder accordingly for clarity. While it would be great to not need this, I currently don't see a way to not have the copilot specifics (apart from extracting it into a separate repository).

It is similar to https://github.com/anomalyco/opencode/pull/5346, but it changes the completions code to properly store the reasoning_opaque field and send it back to the copilot API.

This PR is based on code I wrote for tidewave. I used Claude to implement the same changes based on a fresh copy of the upstream openai-compatible provider: https://github.com/vercel/ai/tree/%40ai-sdk/openai-compatible%401.0.30/packages/openai-compatible/src/chat

There are multiple small commits to make reviewing this easier and I also added tests for the important cases I encountered when handling the reasoning fields.

In the past, the Copilot API failed if the reasoning signature (reasoning_opaque) was not sent back for the Gemini 3 models. At some point GitHub seems to have changed this. Note though that the models still behave differently if the reasoning tokens are not passed back. For example, in Tidewave we've often seen Copilot's Gemini 2.5 spiral into a loop when omitting the reasoning, while it doesn't do that when including those.

How did you verify your code works?

You can chat with Gemini 2.5 Pro (3 Pro Preview is currently broken because of https://github.com/anomalyco/opencode/issues/8829, but it works in the Tidewave version of the code and all the fields are the same).

image

Closes https://github.com/anomalyco/opencode/issues/6864.

SteffenDE avatar Jan 16 '26 15:01 SteffenDE

Hey! Your PR title Add copilot specific provider to properly handle copilot reasoning tokens doesn't follow conventional commit format.

Please update it to start with one of:

  • feat: or feat(scope): new feature
  • fix: or fix(scope): bug fix
  • docs: or docs(scope): documentation changes
  • chore: or chore(scope): maintenance tasks
  • refactor: or refactor(scope): code refactoring
  • test: or test(scope): adding or updating tests

Where scope is the package name (e.g., app, desktop, opencode).

See CONTRIBUTING.md for details.

github-actions[bot] avatar Jan 16 '26 15:01 github-actions[bot]

The following comment was made by an LLM, it may be inaccurate:

Related PRs Found

PR #5346: feat: reasoning text for Gemini 3 Pro in GH Copilot https://github.com/anomalyco/opencode/pull/5346

Why it's related: This PR is explicitly mentioned in the description as similar work. PR #8900 builds on the approach from #5346 but extends it to properly handle reasoning_opaque field storage and transmission for the completions API, whereas #5346 focused on reasoning text for the responses API.


PR #5877: fix(github-copilot): auto-route GPT-5+ models to Responses API https://github.com/anomalyco/opencode/pull/5877

Why it's related: Deals with GitHub Copilot routing and API handling, which is in the same domain as the copilot-specific provider changes.

github-actions[bot] avatar Jan 16 '26 15:01 github-actions[bot]

Thanks for your contribution!

This PR doesn't have a linked issue. All PRs must reference an existing issue.

Please:

  1. Open an issue describing the bug/feature (if one doesn't exist)
  2. Add Fixes #<number> or Closes #<number> to this PR description

See CONTRIBUTING.md for details.

github-actions[bot] avatar Jan 16 '26 15:01 github-actions[bot]

Hello, I can confirm the reasoning works with Gemini 2.5 Pro however it doesn't seem to be the case with GPT 5.2 Codex / GPT 5.2 and Opus 4.5

Coruscant11 avatar Jan 16 '26 19:01 Coruscant11

@Coruscant11 thanks for trying! GPT-5 variants use the responses API version, which this PR does not affect. I need to check Opus - it's possible that reasoning needs to be explicitly enabled.

SteffenDE avatar Jan 16 '26 19:01 SteffenDE

Here's the debugging I did so far -- lmk if there's something wrong with my methodology!

I added basic logging for the opaque signatures 1) received by the API and 2) received by convertToOpenAICompatibleChatMessages. See https://github.com/aadishv/opencode/commit/5776493d4add243cf2bb155ed3989dac835328a9

Then I ran a basic test conversation: image

The logs indicated that convertToOpenAICompatibleChatMessages didn't receive any opaque signatures back (see the "Parsed back:" log)

image

The research I did earlier indicates that this because the AI SDK version currently being used (5.0.x?) doesn't automatically propagate providerMetadata from response -> next request. iirc, we can get this PR to work by upgrading to a version of the AI SDK that does do this, but I don't recall the specifics.

Hope this helps @SteffenDE :)

aadishv avatar Jan 18 '26 22:01 aadishv

I also remember that @​rekram1-node (not pinging rn) was quite knowledgeable on the Copilot API/AI SDK relationship so he might be able to give some insights

aadishv avatar Jan 18 '26 22:01 aadishv

@aadishv the place you log in convertToOpenAICompatibleChatMessages is wrong though, it’s just after initializing the variable so you’ll never see opaque not being undefined there. Try logging here https://github.com/aadishv/opencode/blob/5776493d4add243cf2bb155ed3989dac835328a9/packages/opencode/src/provider/sdk/copilot/chat/openai-compatible-chat-language-model.ts#L320, which is where I tested and did see it working. But as mentioned I’m only on my phone right now - will verify tomorrow :)

I was also using ai-sdk v5 in another project from which I adapted the code and it‘s definitely passing the metadata back.

SteffenDE avatar Jan 18 '26 23:01 SteffenDE

Hmm interesting, added this log: image But I still don't see the signature: image Conversation (the last message is the one where the logs are from): image

Not sure if there's something else wrong with my local setup though. If it worked for you then no worries!

aadishv avatar Jan 18 '26 23:01 aadishv

@aadishv the reasoning_opaque is part of the previous assistant messages, not the user message. So your check would need to be args.messages.at(-2).reasoning_opaque

image

SteffenDE avatar Jan 19 '26 10:01 SteffenDE

@SteffenDE Perhaps you can refer to this https://github.com/caozhiyuan/copilot-api/tree/all/src/routes/messages, which supports claude thinking & interleaved thinking (not native, by prompt), gemini thinking, and gpt responses API. The copilot models API will return model information, such as support for responses API / chat completions / message API (message API may be supported in the next version of claude or the next major version of vscode).

[{
	"billing": {
		"is_premium": true,
		"multiplier": 1,
		"restricted_to": [
			"pro",
			"pro_plus",
			"business",
			"enterprise"
		]
	},
	"capabilities": {
		"family": "gpt-5.2-codex",
		"limits": {
			"max_context_window_tokens": 400000,
			"max_output_tokens": 128000,
			"max_prompt_tokens": 272000,
			"vision": {
				"max_prompt_image_size": 3145728,
				"max_prompt_images": 1,
				"supported_media_types": [
					"image/jpeg",
					"image/png",
					"image/webp",
					"image/gif"
				]
			}
		},
		"object": "model_capabilities",
		"supports": {
			"parallel_tool_calls": true,
			"streaming": true,
			"structured_outputs": true,
			"tool_calls": true,
			"vision": true
		},
		"tokenizer": "o200k_base",
		"type": "chat"
	},
	"id": "gpt-5.2-codex",
	"is_chat_default": false,
	"is_chat_fallback": false,
	"model_picker_category": "powerful",
	"model_picker_enabled": true,
	"name": "GPT-5.2-Codex",
	"object": "model",
	"policy": {
		"state": "enabled",
		"terms": "Enable access to the latest GPT-5.2-Codex model from OpenAI. [Learn more about how GitHub Copilot serves GPT-5.2-Codex](https://gh.io/copilot-openai)."
	},
	"preview": false,
	"supported_endpoints": [
		"/responses"
	],
	"vendor": "OpenAI",
	"version": "gpt-5.2-codex"
},{
    "billing": {
      "is_premium": true,
      "multiplier": 3,
      "restricted_to": [
        "pro",
        "pro_plus",
        "max",
        "business",
        "enterprise"
      ]
    },
    "capabilities": {
      "family": "claude-opus-4.5",
      "limits": {
        "max_context_window_tokens": 160000,
        "max_output_tokens": 16000,
        "max_prompt_tokens": 128000,
        "vision": {
          "max_prompt_image_size": 3145728,
          "max_prompt_images": 5,
          "supported_media_types": [
            "image/jpeg",
            "image/png",
            "image/webp"
          ]
        }
      },
      "object": "model_capabilities",
      "supports": {
        "max_thinking_budget": 32000,
        "min_thinking_budget": 1024,
        "parallel_tool_calls": true,
        "streaming": true,
        "tool_calls": true,
        "vision": true
      },
      "tokenizer": "o200k_base",
      "type": "chat"
    },
    "id": "claude-opus-4.5",
    "is_chat_default": false,
    "is_chat_fallback": false,
    "model_picker_category": "powerful",
    "model_picker_enabled": true,
    "name": "Claude Opus 4.5",
    "object": "model",
    "policy": {
      "state": "disabled",
      "terms": "Enable access to the latest Claude Opus 4.5 model from Anthropic. [Learn more about how GitHub Copilot serves Claude Opus 4.5](https://gh.io/copilot-anthropic)."
    },
    "preview": false,
    "supported_endpoints": [
      "/chat/completions"
    ],
    "vendor": "Anthropic",
    "version": "claude-opus-4.5"
  }]

caozhiyuan avatar Jan 20 '26 02:01 caozhiyuan

I discovered through copilot-api that it's possible to bypass the max_prompt_tokens limit, allowing even Claude models to exceed 200k, though I'm not sure if this is a bug in Copilot. However, I haven't abused this.

caozhiyuan avatar Jan 20 '26 02:01 caozhiyuan

going to test this a bit see how it does

rekram1-node avatar Jan 20 '26 17:01 rekram1-node

Reasoning seems to almost exclusively work for only some gemini models now.. Is that expected? Cant get gemini3 flash to have reasoning, same w/ anthropic models

rekram1-node avatar Jan 20 '26 21:01 rekram1-node

for claude model , should pass header "openai-intent": "conversation-agent" and thinking_budget?: number in ChatCompletionsPayload . for gemini-3-flash , copilot only return thinking signature, not return thinking text.

caozhiyuan avatar Jan 21 '26 01:01 caozhiyuan