FileNotFoundError: Couldn't find a dataset script at /home/BrushNet/webdataset/webdataset.py or any data file in the same directory. Couldn't find 'webdataset' on the Hugging Face Hub either: FileNotFoundError: Dataset 'webdataset' doesn't exist on the Hub

[Tramac]

import os
from datasets import load_dataset

train_data_dir = "./data/BrushData"

train_dataset = load_dataset(
    "webdataset",
    data_files={"train": os.path.join(train_data_dir,"*.tar")},
    split="train",
    streaming=True
)

The data structure is:

|-- data
    |-- BrushData
        |-- 00200.tar
        |-- 00201.tar
        |-- ...

There will be the following error:

FileNotFoundError: Couldn't find a dataset script at /home/BrushNet/webdataset/webdataset.py or any data file in the same directory. Couldn't find 'webdataset' on the Hugging Face Hub either: FileNotFoundError: Dataset 'webdataset' doesn't exist on the Hub

[MikoSamey]

I'm also having the same issue, have you solved it?

[MikoSamey]

I solved it in a very strange way, I created a new environment by following the steps suggested in the documentation, and then the problem was solved. You can try creating a new conda environment...

[Owen718]

Just update your datasets version. I solved it.