shadowleaves

does forward/eval from a trained mamba model require cuda as well?

5

codes in selective_scan_fwd() of selective_scan.cpp seem to suggest even forward from a trained model would require cuda, which might be inconvenient when running models in production environments. Any idea how...