Bedrock fails autocomplete: 'inferenceConfig.stopSequences' failed to satisfy constraint

[mystaroll]

Before submitting your bug report

• [X] I believe this is a bug. I'll try to join the Continue Discord for questions
• [X] I'm not able to find an open issue that reports the same bug
• [X] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: Linux
- Continue: 0.8.52
- IDE: vscode
- Model: any model in AWS Bedrock
- config.json: -

Description

Bedrock provider fails autocomplete with the following

1 validation error detected: Value '[</CO, </COMPLETION>, /src/, #- coding: utf-8, ```, def, class, """#]' at 'inferenceConfig.stopSequences' failed to satisfy constraint: Member must have length less than or equal to 4

I forced an override in config.json with

"completionOptions": {
    "stop": ["</CO"]
  },

but it doesn't do the trick.

To reproduce

No response

Log output

No response

[xytxxx]

• 1, I have the same issue

[xytxxx]

My config.json: ( the problem persists whether or not I include completionOptions)

  "models": [
    {
      "model": "anthropic.claude-3-5-sonnet-20240620-v1:0",
      "provider": "bedrock",
      "title": "Claude 3.5 Sonnet Bedrock",
      "profile": "dev",
      "region": "us-west-2",
      "completionOptions": {
        "temperature": 0.2,
        "topP": 0.9,
        "topK": 40,
        "presencePenalty": 0.2,
        "frequencyPenalty": 0.0,
        "stop": ["\n"],
        "maxTokens": 600
      }
    }
  ],
  "tabAutocompleteModel": {
    "title": "Claude 3.5 Sonnet Bedrock",
    "profile": "dev",
    "region": "us-west-2",
    "model": "anthropic.claude-3-5-sonnet-20240620-v1:0",
    "provider": "bedrock",
    "completionOptions": {
      "temperature": 0.2,
      "topP": 0.9,
      "topK": 40,
      "presencePenalty": 0.2,
      "frequencyPenalty": 0.0,
      "stop": ["\n"],
      "maxTokens": 600
    }
  },

[xytxxx]

It seems to be a bedrock limitation: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent_InferenceConfiguration.html

[malaki12003]

FYI, fixed at https://github.com/continuedev/continue/pull/2554.

[github-actions[bot]]

This issue hasn't been updated in 90 days and will be closed after an additional 10 days without activity. If it's still important, please leave a comment and share any new information that would help us address the issue.

[github-actions[bot]]

This issue was closed because it wasn't updated for 10 days after being marked stale. If it's still important, please reopen + comment and we'll gladly take another look!