champson

GPT2 FP8 don't work on L4/L40 platform

3

### Branch/Tag/Commit main ### Docker Image Version nvcr.io/nvidia/pytorch:23.04-py3 ### GPU name L4 ### CUDA Driver 525.105.17 ### Reproduced Steps ```shell 1. docker run --gpus all -it --rm --ipc=host --ulimit memlock=-1...

[REQUEST]Inference Optimized Pipeline Parallelism

3

As mentioned in the paper https://arxiv.org/abs/2207.00032, DeepSpeed inference supports pipeline parallelism, including hybrid scheduling, offloading activations, and communication, which have led to significant performance improvements. However, does DeepSpeed currently support...