distortion loss impl.

[liruilong940607]

Implement distortion loss for 3DGS from Mip-NeRF 360:

Note: Since the distortion loss is a per-pixel value, I implement it as a per-pixel map rendered from the rasterization() function, with the same resolution with the rendered image. This could be reduced into a scalar with a .mean() call on it for backward. Having the distortion loss as a map instead of a scalar could be useful for visualization and analysis purposes.

[maturk]

@liruilong940607, I think there are some version issues with the Nerfacc library. I have Nerfacc Version: 0.5.3 installed (with pip install nerfacc) however, it seems that this is not compatible with the argument list in your test code. In particular I tracked down the commit and this has not been added to a new release, so pip installing nerfacc results in a version where there is no "indices" arguments to your implementations of inclusive/exclusive sums.

The error a user gets: TypeError: exclusive_sum() got an unexpected keyword argument 'indices'.

This can be fixed by installing from the git source but just a heads up...

[FantasticOven2]

I tried this distortion loss with 2DGS and below are the results: (with distortion loss)

(without distortion loss)

Seems like the improvement is very subtle and focus on details; and this is also the case for the original 2DGS implementation; I will try the L2 version in the 2DGS paper next

[jefequien]

@FantasticOven2 I think the speckle artifacts you have in your normal image are from overflowing uint8 in the visualization. You may need to clamp or normalize your normals.

[keunhong]

Hi Ruilong, do you have any plans to merge this? I think it would be helpful in some more underconstrained settings.

[liruilong940607]

Hi Ruilong, do you have any plans to merge this? I think it would be helpful in some more underconstrained settings.

I haven't got a case where I find it useful and that's why I wasn't quite motivated to spend further efforts on merging it. But I guess it's true that all the cases I tried are well constrained.

I could try to find some cycles to merge this but no guarantee on the timeline!