Wang Yubo comments

Repositories
Issues
Comments

Results 2 comments of


                                            Wang Yubo

gptSessionBenchmark Failed Because of " Assertion failed: d == a + length " with 0.7.1 Release in tritonserver:23.12-trtllm-python-py3 Image

From my experiences, it seems the following combination works well - Build engine using trtllm 0.6.1 - Triton Server 23.11-trtllm-python-py3 to serve the engine

[Feature] Logprobs

> Just to report back, `compute_transition_scores` is exactly what I needed. Thanks for the suggestion. One thing worth mentioning is that, when locating logprobs per token (rather than strings that...