bert.cpp icon indicating copy to clipboard operation
bert.cpp copied to clipboard
verified publisher icon skeskinen

How can I increase n_max_tokens?

Open ztkg257 opened this issue 1 year ago • 1 comments

After modifying the value of n_max_tokens in bert.cpp from "int32_t n_max_tokens = 512;" to "int32_t n_max_tokens = 10000;", I proceeded to rebuild the project. However, upon testing, the value of n_max_tokens remained unchanged at 512 despite the modification. image

ztkg257 avatar Mar 05 '24 14:03 ztkg257

max tokens is a property of the loaded model. Different models might have different values.

skeskinen avatar Mar 05 '24 14:03 skeskinen