WhiteNight

[Question]: paddlenlp_ops安装报错PaddleNLP/csrc/generation/quant_int8.cu(57): error: no suitable user-defined conversion from "__nv_bfloat16" to "__half" exists

7

### 请提出你的问题 运行该代码时报错，PaddleNLP/csrc: python setup_cuda.py install 报错：[2023-11-29 11:14:47,640] [ INFO] spawn.py:38 - /usr/local/cuda/bin/nvcc -I/paddle/lyk/anaconda3/lib/python3.11/site-packages/paddle/include -I/paddle/lyk/anaconda3/lib/python3.11/site-packages/paddle/include/third_party -I/usr/local/cuda/include -I/paddle/lyk/anaconda3/include/python3.11 -I/paddle/lyk/anaconda3/include/python3.11 -c /paddle/lyk/deploy/PaddleNLP/csrc/generation/quant_int8.cu -o /paddle/lyk/deploy/PaddleNLP/csrc/build/paddlenlp_ops/lib.linux-x86_64-cpython-311/quant_int8.cu.o -DPADDLE_WITH_CUDA -DEIGEN_USE_GPU -ccbin cc -Xcompiler -fPIC --expt-relaxed-constexpr -DNVCC...

使用dispatch_model后再generate后报错（Expected all tensors to be on the same device）（参考example中的chat例子）

1

/paddle/lvwenyu01/lyk/model_dir/InternLM/modeling_InternLM.py:1745 in _reorder_cache │ │ │ │ 1742 │ │ │ for past_state in layer_past: │ │ 1743 │ │ │ │ │ │ 1744 │ │ │ │ print(past_state.device)...

l can't get the required file from the data.py, include 'retailrocket-view.inter' and 'gowalla.inter'. so l replace the filename with row filename l got from the online. But the statistics are...