mofanke
mofanke
disable cache_prompt works
I had the same problem, c.NotebookNotary.data_dir by the docs can't work, I tried environment variable:JUPYTER_DATA_DIR, it worked. but it's very confused that what the docs says did not work
> We added a mamba-codestral-7B-v0.1 exampel in today's update. Please refer to https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/mamba and have a try. cannot install tensorrt_llm==0.12.0.dev2024072301
> > cannot install tensorrt_llm==0.12.0.dev2024072301 > > You need to reinstall tensorrt_llm. convert ok, but trtllm-build failed [TensorRT-LLM] TensorRT-LLM version: 0.12.0.dev2024072301 [07/25/2024-14:40:30] [TRT-LLM] [W] Implicitly setting PretrainedConfig.layer_types = ['recurrent'] [07/25/2024-14:40:30]...
> https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/mamba sorry, i start a new python env and it works. thx for that , i will close the issue.
@dacohen We’ve also encountered a similar issue. Although we can work around it using regex, we still hope that this behavior can be fixed directly. I think your method is...
 works for me