leachee99

遇到了同样的问题，请问有大佬解决了吗？

hi, have DeepSpeed supported Mixtral(or other Moe models) with the old inference interface ? I tried to inference the Moe model(mixtral and Qwen1.5-MoE-A2.7B) with DeepSpeed in multi-Node, but it failed....

I am experiencing the same issue with enabling TP+EP for inference on the Mixtral model（or other MoE model）. Have you managed to find a solution to this problem?

> > I am experiencing the same issue with enabling TP+EP for inference on the Mixtral model（or other MoE model）. Have you managed to find a solution to this problem?...

> > > > I am experiencing the same issue with enabling TP+EP for inference on the Mixtral model（or other MoE model）. Have you managed to find a solution to...

> > > > > > I am experiencing the same issue with enabling TP+EP for inference on the Mixtral model（or other MoE model）. Have you managed to find a...

from opencompass.configs.datasets.demo.demo_gsm8k_base_gen import gsm8k_datasets from opencompass.configs.datasets.collections.base_medium_llama import piqa_datasets, siqa_datasets from opencompass.configs.datasets.hellaswag.hellaswag_gen_6faab5 import hellaswag_datasets from opencompass.configs.datasets.ARC_e.ARC_e_gen_1e0de5 import ARC_e_datasets from opencompass.configs.datasets.commonsenseqa.commonsenseqa_gen_1da2d0 import commonsenseqa_datasets 我尝试了以上的数据集，其中gsm8k_datasets是可以的， 但其他的都显示TypeError: Value after * must be an iterable,...