Liger-Kernel icon indicating copy to clipboard operation
Liger-Kernel copied to clipboard
Published 1 month ago • verified publisher icon linkedin

DeepSeek Native Sparse Attention (NSA) Kernel

Open qingquansong opened this issue 10 months ago • 6 comments

🚀 The feature, motivation and pitch

Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention https://arxiv.org/abs/2502.11089

Potentially useful python reference https://github.com/dhcode-cpp/NSA-pytorch

Alternatives

No response

Additional context

No response

qingquansong avatar Apr 05 '25 06:04 qingquansong

Would like to take this up!

shivam15s avatar Apr 05 '25 21:04 shivam15s

I'm interested in it. Is there anything I can help?

Tcc0403 avatar Apr 07 '25 09:04 Tcc0403

Hi, I want to join this work as well. Is there any ongoing progress? Thanks!

mRSun15 avatar Apr 11 '25 17:04 mRSun15

Any updates / active branches on this? @Tcc0403 @shivam15s

AndreSlavescu avatar Jun 09 '25 19:06 AndreSlavescu

@mRSun15 @AndreSlavescu I'm not working on it. Feel free to pick it up!

Tcc0403 avatar Jun 09 '25 20:06 Tcc0403

@mRSun15 @AndreSlavescu I'm not working on it. Feel free to pick it up!

@mRSun15 Are you working on this / Do you want to pick this up? If not, I can pick it up once I have more cycles @Tcc0403

AndreSlavescu avatar Jun 10 '25 16:06 AndreSlavescu