Error of fused_dense module of flash_attn

[Jenny199099]

Hi author, I have compiled and installed fused_dense_lib successfully. But when I tried to run the finetuning code, I encountered this error: "RuntimeError: linear_act_forward failed." which is due to the line 291 in InternVideo2/single_modality/models/internvideo2.py. The complete error is as shown below:

Traceback (most recent call last): File "", line 1, in File "/nvme/miniconda3/envs/internvideo/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl return forward_call(*input, **kwargs) File "/nvme/miniconda3/envs/internvideo/lib/python3.8/site-packages/flash_attn/ops/fused_dense.py", line 457, in forward out = fused_mlp_func( File "/nvme/miniconda3/envs/internvideo/lib/python3.8/site-packages/flash_attn/ops/fused_dense.py", line 391, in fused_mlp_func return FusedMLPFunc.apply( File "/nvme/miniconda3/envs/internvideo/lib/python3.8/site-packages/torch/cuda/amp/autocast_mode.py", line 97, in decorate_fwd return fwd(*args, **kwargs) File "/nvme/miniconda3/envs/internvideo/lib/python3.8/site-packages/flash_attn/ops/fused_dense.py", line 257, in forward output1, *rest = fused_dense_cuda.linear_act_forward( RuntimeError: linear_act_forward failed.

Could you please help me to fix this? Thank you very much.

[Andy1621]

I have not met this problem. Maybe you can refer to this issue https://github.com/Dao-AILab/flash-attention/issues/289#issuecomment-1620944622.

[Jenny199099]

I have not met this problem. Maybe you can refer to this issue Dao-AILab/flash-attention#289 (comment).

Thank you for your reply. I follow the above comment to not use fused_mlp with DeepSpeed. This can fix this error.