Andreas Kieslinger issues

Repositories
Issues
Comments

Results 1 issues of


                                            Andreas Kieslinger

Overlap CUDA graph building and processing to minimize GPU idle time and improve tokens per seconds performance.

Hi all, this PR takes the ideas applied to the vulkan backend (#9118 and #10499) and implements them for CUDA. This results in improved tokens per second performance. ### Performance...

Nvidia GPU

ggml