Perry Zou

> I am trying to compile on Mac running Big Sur 11.2.3 and getting > qemu-system-riscv64: qemu_mprotect__osdep: mprotect failed: Permission denied > > Any idea how to get around this....

I think that there isn't currently an exposed interface for the 'trust_remote_code' parameter. You can set it by modifying the load method directly in the deepspeed source code. [https://github.com/microsoft/DeepSpeed/blob/2a6c58df34ea04e0aa451a4623b7c7d49b087a50/deepspeed/inference/v2/checkpoint/huggingface_engine.py#L24](https://github.com/microsoft/DeepSpeed/blob/2a6c58df34ea04e0aa451a4623b7c7d49b087a50/deepspeed/inference/v2/checkpoint/huggingface_engine.py#L24)

refer to #4913 Install DeepSpeed from the latest source code and consider utilizing DeepSpeed-MII for optimal performance.

@rayquazaMega I have not used the VL model, I think it's not well-supported currently. For Chat or Base model, I would like recommend `deepspeed-mii`, it's describled in https://github.com/microsoft/DeepSpeed/tree/master/blogs/deepspeed-fastgen and https://github.com/microsoft/DeepSpeed-MII

Hi @freQuensy23-coder, I submitted a PR to DeepSpeed to support Qwen1.5-MoE, and it's now waiting for deepspeed repo merge. before that, you can build deepspeed manually from my source code....

Hi, you can refer these docs and code examples: - https://github.com/microsoft/DeepSpeed/tree/master/blogs/deepspeed-fastgen - https://github.com/microsoft/DeepSpeedExamples/tree/master/inference/mii for adding new unsupported models: - https://github.com/microsoft/DeepSpeed/blob/master/deepspeed/inference/v2/model_implementations/AddingAModel.md for loading local huggingface checkpoints, you can specify the absolute...