dwq370

采用自己的数据微调INT4模型，使用web_demo部署后推理，问一个问题在queue等待了190s后没有返回任何结果，请教大佬们这是什么原因导致的

9

### Is there an existing issue for this? - [X] I have searched the existing issues ### Current Behavior  ### Expected Behavior _No response_ ### Steps To Reproduce 1、用自己的数据微调INT4模型...

3090显卡，CUDA11.1版本，单卡运行INT4推理报错

8

3090显卡，CUDA11.1版本，单卡运行INT4推理报错  是CUDA版本的问题吗？MOSS最低CUDA版本是哪个？

launch multi-gpu triton server and got an Error

1

### System Info 4*NVIDIA L20 ### Who can help? _No response_ ### Information - [X] The official example scripts - [ ] My own modified scripts ### Tasks - [...

request is blocked and non output when using tensor parallelism with multi gpus

7

### System Info NVIDIA 2*L20 launch triton server with tensorrt-llm backend v0.12.0 in a container ### Who can help? _No response_ ### Information - [ ] The official example scripts...

### System Info Ubuntu 22.04 Triton image: nvcr.io/nvidia/tritonserver:24.06-trtllm-python-py3 and the version of trtllm-backend is 0.10.0 Model: qwen2-7b-instruct ### Who can help? _No response_ ### Information - [ ] The official...

ModuleNotFoundError: No module named 'pllm_python' when serve_8B.py executed

1

step1: launch container ``` mkdir -p ~/nano-test docker run --gpus all --net=host --privileged -v /dev/shm:/dev/shm --name nanoflow -v ~/nano-test:/code -it nvcr.io/nvidia/nvhpc:23.11-devel-cuda_multi-ubuntu22.04 ``` step2: Install dependencies ``` git clone https://github.com/efeslab/Nanoflow.git cd...