Chat responses containing LaTeX are occasionally truncated

[steventsvetkov]

Before submitting your bug report

• [X] I believe this is a bug. I'll try to join the Continue Discord for questions
• [X] I'm not able to find an open issue that reports the same bug
• [X] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: Pop!_OS 22.04 LTS x86_64
- Continue: v0.9.184 (pre-release)
- IDE: VSCode 1.91.0
- Model: GPT-4o
- config.json:
  
{
  "models": [
    {
      "apiKey": "",
      "completionOptions": {
        "temperature": 0,
        "maxTokens": 4092
      },
      "contextLength": 128000,
      "model": "gpt-4o",
      "provider": "openai",
      "systemMessage": "You give helpful and concise responses about math and machine learning. You are an AI language model that always uses LaTeX for mathematical expressions. Whenever you include equations or variables, ensure they are enclosed in $ signs. For example, f = wx + b should be written as $f = wx + b$ and \\mathbf{y} should be written as $\\mathbf{y}$. Prefer shorter responses and explain any technical jargon you use. Explain the concepts in simple words, while also additionally providing the technical jargon.",
      "title": "GPT-4o Math"
    }
  ]
}

Description

Chat responses are occasionally cut off. I've noticed this happening when the response contains LaTeX expressions, wrapped in $ for proper rendering.

This happens around one in very 15 responses with math expressions and I can't see a pattern as to why it sometimes works and other times it doesn't. The cutoff point can be right before $ or $$ characters but it can also happen in the middle of a normal sentence. There is nothing being logged in the console or devtools when this happens.

I can still get the full chat response by using the copy button or in the output log, it seems to be a rendering issue.

To reproduce

1. Set a system prompt instructing the model to produce LaTeX expressions and warp them in $ for proper rendering.
2. Ask the model to explain something math related.
3. Most of the time it will work, but occasionally the responses get cut off in the UI

Log output

==========================================================================
==========================================================================
Settings:
contextLength: 128000
temperature: 0
maxTokens: 4092
model: gpt-4o
log: undefined

############################################

<system>
You give helpful and concise responses about math and machine learning. You are an AI language model that always uses KaTeX for mathematical expressions. Whenever you include equations or variables, ensure they are enclosed in $ signs. For example, f = wx + b should be written as $f = wx + b$ and \mathbf{y} should be written as $\mathbf{y}$. Prefer shorter responses and explain any technical jargon you use. Explain the concepts in simple words, while also additionally providing the technical jargon.

<user>
What are the roots of unity?

==========================================================================
==========================================================================
Completion:

The roots of unity are special complex numbers that, when raised to a certain positive integer power, equal 1. 

In technical terms, the $n$-th roots of unity are the solutions to the equation $z^n = 1$, where $z$ is a complex number and $n$ is a positive integer.

These roots can be expressed as:
$$
z_k = e^{2\pi i k / n} \quad \text{for} \quad k = 0, 1, 2, \ldots, n-1
$$

Here, $e$ is the base of the natural logarithm, $\pi$ is the mathematical constant Pi, and $i$ is the imaginary unit with $i^2 = -1$. The term $2\pi i k / n$ represents the angle in radians.

In simpler terms, the $n$-th roots of unity are points on the complex plane that are evenly spaced around the unit circle.

[Jiayou-Chao]

The issue happens when latex equations are wrapped by $$. For example:

   $$ 
   x' = \frac{x - \text{min}(X)}{\text{max}(X) - \text{min}(X)} 
   $$

$$equation$$ and \(equation\) are fine.

$$
equation
$$

will cause this issue.

You can use the copy button or set

  "ui": {
    "displayRawMarkdown": false
  }

in config.json to see the full message.

[diogovalada]

Is continue not able to represent latex equations?

[sestinj]

To summarize what has been discovered in this thread:

Continue will only display LaTeX when it is wrapped in dollar signs, and will cause the rest of the output to be truncated when wrapped in double dollar signs.

If anyone is interested in further investigating this, we are using rehype/remark plugins here: https://github.com/continuedev/continue/blob/f19cec4e0944773142b3d6414e2b76f38bcf3f66/gui/src/components/markdown/StyledMarkdownPreview.tsx#L156-L158

[RomneyDa]

I opened a separate issue for display issues here with several thoughts and tinkering observations, thanks for the input @steventsvetkov, @Jiayou-Chao, @Astlaan

https://github.com/continuedev/continue/issues/3068

The specific truncation issue should be fixed by #3150

@Astlaan if you ask it to use "inline double dollar sign syntax with no line breaks around the dollar signs" it should work

[RomneyDa]

Fixed by #3150

Discussion for other issues like not supporting parentheses and brackets continued here https://github.com/continuedev/continue/issues/3068