Shanshan Shen issues

Repositories
Issues
Comments

Results 2 issues of


                                            Shanshan Shen

[Misc] Add get_stream_cls() method for Platform class

# What does this PR do? Add `get_stream_cls()` method for `Platform` class, making the code related to Stream creating decoupled with specific hardware, e.g., `cuda`.

speculative-decoding

[Bugfix] Make the fused_moe code compatible with non-triton supported hardware

# What does this PR do? When running vLLM with `MiniCPM` model (not moe), it will get an error shown that `triton` is not supported on some devices. Thus, I...