Bug in diffusers basic training tutorial

[yue-here]

Describe the bug

In the diffusers basic training tutorial, setting the manual seed argument (generator=torch.manual_seed(config.seed)) in the pipeline call inside evaluate() function rewinds the dataloader shuffling, leading to overfitting due to the model see same sequence of training examples after every evaluation call. This can be tested by running the code without this argument. In the former case the training loss is lower but quality of sampled images are worse to the human eye.

images = pipeline(
        batch_size=config.eval_batch_size,
        generator=torch.manual_seed(config.seed),
    ).images 

Reproduction

images = pipeline( batch_size=config.eval_batch_size, # generator=torch.manual_seed(config.seed), # Dataloader shuffling works fine once this is removed ).images

Logs

No response

System Info

• diffusers version: 0.28.0.dev0
• Platform: Windows-10-10.0.22631-SP0
• Python version: 3.11.9
• PyTorch version (GPU?): 2.1.2 (True)
• Huggingface_hub version: 0.22.2
• Transformers version: 4.40.1
• Accelerate version: 0.29.3
• xFormers version: not installed

Who can help?

@stevhliu