Hongjie1Chu

Question with forward_backward_pipelining_without_interleaving in Megatron-LM Pipeline

1

I encountered a problem when using the Megatron pipeline. The function I am using is forward_backward_pipelining_without_interleaving. In this pipeline function, each pipeline stage calls forward_step for the forward pass: output_tensor...

Title: CUDA RuntimeError: Unspecified Launch Failure during Training

10

### System Info - `transformers` version: 4.41.0 - Platform: Linux-5.15.0-88-generic-x86_64-with-glibc2.35 - Python version: 3.10.6 - Huggingface_hub version: 0.23.0 - Safetensors version: 0.4.3 - Accelerate version: 0.27.2 - Accelerate config: not...