Hendrik Weiske comments

Repositories
Issues
Comments

Results 3 comments of


                                            Hendrik Weiske

Invalid MIT-MAGIC-COOKIE-1 key

I am getting the same error trying to build mistral for ChatRTX on linux using `python build.py --model_dir './model/mistral/mistral7b_hf' --quant_ckpt_path './model/mistral/mistral7b_int4_quant_weights/mistral_tp1_rank0.npz' --dtype float16 --remove_input_padding --use_gpt_attention_plugin float16 --enable_context_fmha --use_gemm_plugin float16 --use_weight_only...

Docs() Conflicts with LiteLLM Models

Yes. same issue.

settings are correctly finding the locally running model, but docs gives err authentication error

The API error results from the open API key. It seems that the docs object has a problem with the specification of no API key, due to the local model.