Experiments on extrapolation with diffusion

[dtch1997]

Goal

We're trying to answer the question, can diffusion models denoise beyond the original level of noise in the dataset?

Results

Extrapolating the time step past 0 doesn't seem to work on CIFAR-10.

Need to check whether this is something hardcoded due to e.g. alpha, beta scheduling.

Time step 0:

Time step -1:

It may be easier to start with a synthetic dataset and go from there.