chenyaofo

Large size of the tar files generated by TarWriter

2

- Problem I try transforming the dataset [ImageNet-C](https://drive.google.com/drive/folders/1HDVw6CmX3HiG0ODFtI75iIfBDxSiSz2K) (an image classification dataset) into webdataset tarfile formats. The original dataset includes 4 tar files that store image samples. The size of...

[BUG] Fail to compile Deepspeed 0.9.0 with CUDA 11.7 and PyTorch 1.13.1 with Docker.

12

**Describe the bug** I am building a docker image via Github Action, I installed pytorch 1.13.1 with cuda 11.7. Then when I am trying to install deepspeed 0.9.0 by `DS_BUILD_OPS=1...

**Describe the Question** I just notice code in https://github.com/HuangLK/llama-deepspeed/blob/faedea514b11c18c695e1b2a6adb63b102ef001c/models/llama_pipeline_model.py#L174 It appears that the code is utilizing the `PipeModelDataParallelTopology` API. I'm curious to know if it supports model parallelism during training....

How to split web dataset shards in parallel external source with worker id in DALI?

13

### Describe the question. I seek to write external source to load webdatasets from S3. And I want to adopt the parallel external source to accelerate the loading. Usually, the...

Firstly, I would like to express my sincere gratitude to all the developers for creating and maintaining Sedna, an outstanding AI cloud-edge collaboration project. I am currently utilizing the Sedna...