zigma icon indicating copy to clipboard operation
zigma copied to clipboard
Published 10 months ago • verified publisher icon CompVis

Any code and how to change the dataset to webdataset format?

Open huangjch526 opened this issue 1 year ago • 2 comments

huangjch526 avatar Aug 26 '24 12:08 huangjch526

I want to make the webdataset format of MSCOCO dataset, could you tell me how to make that in your code?

huangjch526 avatar Aug 26 '24 12:08 huangjch526

You can follow the READMe in this repo, they also use webdataset.

https://github.com/bytedance/1d-tokenizer?tab=readme-ov-file

dongzhuoyao avatar Aug 30 '24 21:08 dongzhuoyao