(intellij) tab completion outputs too verbose completion "it looks like you are coding(...)" instead of actual code

[laurentperez]

Before submitting your bug report

• [X] I believe this is a bug. I'll try to join the Continue Discord for questions
• [X] I'm not able to find an open issue that reports the same bug
• [X] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS:Ubuntu 22.04.4 LTS
- Continue:0.0.62
- IDE:idea
- Model:llama3.1:latest
- config.json:
  
(...)
  "tabAutocompleteModel": {
    "title": "Tab",
    "provider": "openai",
      "model": "llama3.1:latest",
      "apiBase": "http://xxxx:11434"
  },
  "embeddingsProvider": {
    "provider": "ollama",
    "model": "nomic-embed-text" # using local ollama not remote
  }

Description

• I'm using a remote ollama server hosted on xxxx using llama 3.1
• I understand this is not using codestral nor starcoder
• I've read and understood https://docs.continue.dev/features/tab-autocomplete#setting-up-a-custom-model

To reproduce

what happens :

• I code, then I tab
• tab autocompletes but it does not complete with code, it completes with a "human guided" output like "it looks like you are coding(...)"
• I'd prefer to complete with a code block, and not a verbose description of what the LLM understands about my current state

how could I set a custom systemMessage for the tab completion ? I'd like something similar to this : "just output the code, be concise".

setting systemMessage in tabAutocompleteModel seems to have no effect at all.

is it even the right scenario here, or instead of llama3.1 should I just use starcoder ? I can't use codestral, I don't want my prompts to be sent to Mistral.

Log output

No response

[bitbottrap]

You may be fighting an uphill battle against the model. Since you're using this for completion I'm guessing you're using the 8B model for responsiveness. You might not be able to consistently get the output you want even by being more specific in the prompting.

[sestinj]

@laurentperez systemMessage does not apply to autocomplete, as the prompt has to be very specific and doesn't have room for modification.

Unfortunately llama 3 is intended as a chat model rather than autocomplete, so you shouldn't expect great results. I would recommend trying deepseek-coder:6.7b instead https://ollama.com/library/deepseek-coder

[laurentperez]

yep I understood when reading the template selector https://github.com/continuedev/continue/blob/main/core/autocomplete/templates.ts#L314

"llama" will not match the includes, "deepseek" will and obv. as @sestinj pointed out, llama is intended as a chat model anyway. GH copilot autocompletion uses Codex but with a highly specific autocompletion prompt too.

I'll report and close the issue when I test deepseek or another model intended for completion.

[diwakar-s-maurya]

If anyone is trying to use continue.dev with openwebui, the model produces verbose autocompletes if the provider is "openai". Changing the provider to "ollama" helps. I found this happening with qwen2.5-coder:1.5b

Example: This configuration provides the code autocompletion only without verbose sentences.

{
  "tabAutocompleteModel": {
      "title": "qwen2.5-coder:1.5b",
      "provider": "ollama",  // <-------
      "model": "qwen2.5-coder:1.5b",
      "useLegacyCompletionsEndpoint": false,
      "multilineCompletions": "always",
      "apiBase": "http://YOUROPENWEBUI/ollama",
      "apiKey": "API_KEY"
    }
}

[github-actions[bot]]

This issue hasn't been updated in 90 days and will be closed after an additional 10 days without activity. If it's still important, please leave a comment and share any new information that would help us address the issue.

[github-actions[bot]]

This issue was closed because it wasn't updated for 10 days after being marked stale. If it's still important, please reopen + comment and we'll gladly take another look!