Langlin Huang

icon indicating a website link https://shrango.github.io/

icon location First-year PhD student at Washington University in St. Louis. Obtained Master's degree at Institute of Computing Technology, Chinese Academy of Sciences

Results 1 issues of Langlin Huang

[report bug] Encountered when inferencing with Mistral models

I met the following error when inferencing with base_model `Mistral-7b-Instruct-v0.2` File "~/Medusa/medusa/model/modeling_mistral_kv.py", line 74, in _make_sliding_window_causal_mask mask = torch.triu(mask, diagonal=-sliding_window) ^^^^^^^^^^^^^^^ TypeError: bad operand type for unary -: 'NoneType' I...