hao-ai-lab
[Bug] The calculation of 'cur_topk'
Describe the bug
Hello @SolitaryThinker ,
I found here using attn_metadata.total_seq_length to calculate the top-k blocks:
https://github.com/hao-ai-lab/FastVideo/blob/50da62e722165a8847895a551aa56bc5ee2bb08c/fastvideo/attention/backends/video_sparse_attn.py#L271
where total_seq_length was calculated by:
https://github.com/hao-ai-lab/FastVideo/blob/50da62e722165a8847895a551aa56bc5ee2bb08c/fastvideo/attention/backends/video_sparse_attn.py#L177
but after the preprocess function, sequence are padded hence length might be increased, right?
https://github.com/hao-ai-lab/FastVideo/blob/50da62e722165a8847895a551aa56bc5ee2bb08c/fastvideo/attention/backends/video_sparse_attn.py#L228
So will the current calculation of 'cur_topk' using a smaller vaule (<= the padded one) as the upper bound?
I’d really appreciate any clarification.
