Yu-Hsiang Wang issues

Repositories
Issues
Comments

Results 2 issues of


                                            Yu-Hsiang Wang

Add on-paper form of RoPE kernel

## Summary Implement the on-paper form of the RoPE kernel from [RoFormer](https://arxiv.org/pdf/2104.09864.). This implementation does not support optional value input, unlike the HuggingFace [RoFormer RoPE](https://github.com/huggingface/transformers/blob/v4.46.0/src/transformers/models/roformer/modeling_roformer.py#L309) implementation. ## Details The code...

Add support for GPT-OSS models

## Summary Add GPT-OSS model support, addressing https://github.com/linkedin/Liger-Kernel/issues/848 Completed patching for RoPE, RMSNorm, cross_entropy, and fused_linear_cross_entropy. ### Known Issues - **Gated SwiGLU Patching Support**: The current Hugging Face implementation of...