Xiang Ying

[BUG] Wrong output for batch input for opt model inference.

4

**Describe the bug** I got a question when I use batch inference to opt model. Just the first output of the batch results is right. The last output are all...

CUBLAS_STATUS_INTERNAL_ERROR for OPT-13b when input token length > 230 (with generate token length = 20)

1

### Description ```shell Master Branch, V100 GPU. GPU Driver Version: 470.82.01 CUDA Version: 11.7 ``` ### Reproduced Steps ```shell 1. download opt-13b weight from huggingface. 2. convert the weight to...