Ivan Mihić issues

Repositories
Issues
Comments

Results 1 issues of


                                            Ivan Mihić

Inference hangs after running Llama 3.1 8B engine built with either TP=4 or PP=4 but works ok if built with TP=1

### System Info **System information** Lenovo SR675V3 CPU: 2x AMD Epyc 9454 CPU MEMORY: 502GB GPU: 4x Nvidia L40s (all connected through PCIE slots to one of the two available...

bug