TensorRT-LLM icon indicating copy to clipboard operation
TensorRT-LLM copied to clipboard
verified publisher icon NVIDIA

refactor: Replace DecoderFinishedEvent with CudaEvent in decoder classes

Open Funatiq opened this issue 1 year ago • 7 comments

  • Updated the forwardAsync method in GptDecoderBatched and iGptDecoderBatched to return CudaEvent instead of DecoderFinishedEventPtr, simplifying event handling.
  • Removed the DecoderFinishedEvent class and its associated usage across various files, streamlining the codebase.
  • Adjusted related methods and Python bindings to accommodate the new event structure, ensuring compatibility and maintaining functionality.

These changes enhance the clarity and efficiency of the decoding process in the batch manager.

Funatiq avatar Mar 25 '25 21:03 Funatiq

/bot run

Funatiq avatar Mar 25 '25 21:03 Funatiq

PR_Github #471 [ run ] triggered by Bot

niukuo avatar Mar 25 '25 21:03 niukuo

PR_Github #471 [ run ] completed with state SUCCESS /LLM/main/L0_MergeRequest_PR pipeline #404 completed with status: 'FAILURE'

niukuo avatar Mar 25 '25 23:03 niukuo

/bot run

Funatiq avatar Mar 26 '25 07:03 Funatiq

PR_Github #537 [ run ] triggered by Bot

niukuo avatar Mar 26 '25 07:03 niukuo

PR_Github #537 [ run ] completed with state SUCCESS /LLM/main/L0_MergeRequest_PR pipeline #458 completed with status: 'SUCCESS'

niukuo avatar Mar 26 '25 09:03 niukuo

Looks good to me (can't approve, I would if I could).

dcampora avatar Mar 26 '25 21:03 dcampora

/bot reuse-pipeline

Funatiq avatar Mar 28 '25 06:03 Funatiq

PR_Github #674 [ reuse-pipeline ] triggered by Bot

tensorrt-cicd avatar Mar 28 '25 06:03 tensorrt-cicd

PR_Github #674 [ reuse-pipeline ] completed with state SUCCESS Reusing PR_Github #537 for commit 8408e3b

tensorrt-cicd avatar Mar 28 '25 06:03 tensorrt-cicd