optional max_tokens

[Algorithm5838]

Use a checkbox to optionally enable the use of max_tokens instead of having it disabled. This feature is useful for OpenAI models, as well as models from OpenRouter and other platforms. I've set the default to 2048 for smaller context models (4k); however, 4096 is the preferred setting for newer models from OpenAI and Anthropic. Despite these models supporting much larger contexts, their output is capped at 4096.

[vercel[bot]]

@Algorithm5838 is attempting to deploy a commit to the NextChat Team on Vercel.

A member of the Team first needs to authorize it.

[github-actions[bot]]

Your build has completed!

Preview deployment

[H0llyW00dzZ]

Use a checkbox to optionally enable the use of max_tokens instead of having it disabled. This feature is useful for OpenAI models, as well as models from OpenRouter and other platforms. I've set the default to 2048 for smaller context models (4k); however, 4096 is the preferred setting for newer models from OpenAI and Anthropic. Despite these models supporting much larger contexts, their output is capped at 4096.

@Algorithm5838 Just letting you know, there is a bug related to the attach messages feature due to the max_tokens setting in this chat.ts file. The logic needs to be refactored because the way attach messages work is not consistent, depending on the max_tokens value.

[H0llyW00dzZ]

related issue:

• #4303

[Algorithm5838]

You are correct. I encountered it before and solved it by commenting out this part:

          i >= contextStartIndex;// && tokenCount < maxTokenThreshold;

The issue with the logic is that they assumed max_tokens is input + output, where it is actually output only. The right way is to include context tokens with the models.

[H0llyW00dzZ]

You are correct. I encountered it before and solved it by commenting out this part:

          i >= contextStartIndex;// && tokenCount < maxTokenThreshold;

The issue with the logic is that they assumed max_tokens is input + output, where it is actually output only. The right way is to include context tokens with the models.

I figured that out a few weeks ago when trying to implement support for anthropic with my friends.