Rajdeep Borgohain

langchain_nvidia_trt not working

3

I have gone through the notebooks but couldn't able to stream the tokens from the TensorRTLLM. Here's the issue:  Code used: ```python from langchain_nvidia_trt.llms import TritonTensorRTLLM import time import...

[BUG] Issue serving Qwen/Qwen1.5-14B-Chat using vLLM OpenAI Client(Online serving)

1

### 是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this? - [X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions ### 该问题是否在FAQ中有解答？ | Is there an...

Failed to convert microsoft/Phi-3-medium-128k-instruct

1

NotImplementedError: RoPE scaling type 'longrope' is not yet implemented. The following RoPE scaling types are currently supported: linear, su, llama3

**Describe the bug** I am trying to install and use the model on a aarch64 machine but getting issues because of the onnx-runtime. Any suggestions?