YC

icon company Ant Group icon location Interested in LLM Inference

Results 3 comments of YC

Error/Crash running on macOS Apple M1

same error on macOS Intel

[Feature Request] llama v3 support

I think it is because of the vocab_size difference between llama2 and 3 (32000 vs 128256) Somehow inside TRT-LLM there seems to be an pre-defined shape when we initialize the...

Error when running torchtune qwen3 MoE no grouped_mm on SM89

Hi! A little bit off but I am just a little curious about the tp_plan of qwen3 since I cannot find this model component under ./torchtune/models/qwen2 or qwen3. Did you...