EUfON
EUfON
Same issue
Video card list of TCNN_CUDA_ARCHITECTURES: https://docs.nerf.studio/quickstart/installation.html
It happened again
I'm trying using flash attention in modelscope-agent, which needs layer_norm and rotary.Now flash attention and rotary has been built by @bdashore3 's branch, while layer_norm in error. I used py3.10,...
> You don't have to use layer_norm. However, I made it work. The trouble is in ln_bwd_kernels.cuh [line 54](https://github.com/Dao-AILab/flash-attention/blob/02ac572f3ffc4f402e4183aaa6824b45859d3ed3/csrc/layer_norm/ln_bwd_kernels.cuh#L517) For some reason unknown, BOOL_SWITCH not worked as turning bool has_colscale...