dataloader icon indicating copy to clipboard operation
dataloader copied to clipboard

PyTorch Loader not working

Open mohammad-qloo opened this issue 2 years ago • 1 comments

I tried the Pytorch loader but it is giving me the following error:

BufferError: DLPack only supports signed/unsigned integers, float and complex dtypes.

but if I switched Tensorflow it works. Here's the code snippet:

train_dataset = merlin.io.Dataset(train_path, engine="parquet")
from merlin.loader.torch import Loader
# from merlin.dataloader.tensorflow import Loader
dl = Loader(train_dataset, batch_size=4096)

mohammad-qloo avatar Oct 16 '23 20:10 mohammad-qloo

Hello @mohammad-qloo, can you give us some further information about the data you are using in the loader. From the error you posted it seems this may be a dtype issue. Where it is possible that tensorflow is auto converting but pytorch is not.

jperez999 avatar Oct 31 '23 19:10 jperez999