RobotGF

CN Deep Learning+HPC Pytorch CUDA

Results 3 comments of


                                            RobotGF

GPTNeox decoding argumentation

"repetition_penalty": 1.01 have bug

Detected layernorm nodes in FP16.

> This message seems to go away when using bfloat16 fwiw Maybe It only detect FP16?

[TensorRT-LLM][ERROR] Assertion failed: mNextBlocks.empty() (/home/jenkins/agent/workspace/LLM/main/L0_PostMerge/llm/cpp/tensorrt_llm/batch_manager/kvCacheManager.cpp:160)

I also met this problem on branch v0.7.1 But the main branch now is OK