artifacts in planar projection images

[dialectrics]

Describe the bug When extracting planar projections from equirectangular images there are strange artifacts in the results not present in the originals.

Original image: Two example planar projection outputs: Notice the artifacts around the horizon line and on the guard rail. Issue not isolated to just this dataset. Here's an example from a different video. Compare this with the same planar projection made using an earlier version of nerfstudio.

To Reproduce

1. run ns-process-data images --camera-type equirectangular --images-per-equirect 8 --data path/to/data --output-dir processed/
2. Inspect images in "planar projections" folders for artifacts

Expected behavior Planar projection images without artifacts

Additional context I recently reinstalled conda and nerfstudio from scratch on a new machine and built from the git repo rather than pip install. There is a command line warning that pops up that I never saw previously when running ns-process-data. I went into the "functional.py" script and changed align_corners=True from false due to even worse artifacts appearing without that change. Maybe this is a user error but I have never seen such an error pop up and did my best to mitigate it. Any input appreciated, I can share datasets but it should happen with any equirectangular image in my experience. Cheers.

[14:19:06] Generating 8 (1440, 1440) sized images per equirectangular image           images_to_nerfstudio_dataset.py:55
/home/mrpibb/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/torch/nn/fun
ctional.py:4236: UserWarning: Default grid_sample and affine_grid behavior has 
changed to align_corners=False since 1.3.0. Please specify align_corners=True if
the old behavior is desired. See the documentation of grid_sample for details.
  warnings.warn(

[tancik]

My guess is that this is due to a change in PR #2350. This change was required for licensing reasons. @THE-COB could also chime in.

[THE-COB]

My guess is that this is due to a change in PR #2350. This change was required for licensing reasons. @THE-COB could also chime in.

This is probably true, let me see what I can do about the artifacts

[lolo2k]

Hi, any update on this because i have reinstalled Nerfstudio just now and still i get the same artifacts. And a lot of them.

[THE-COB]

Hi, any update on this because i have reinstalled Nerfstudio just now and still i get the same artifacts. And a lot of them.

Sorry, I have not gotten to it yet. I will update you when I get to it or someone else does.

[lolo2k]

Thanks for the update

On Tue, 31 Oct 2023, 16:18 Rohan Mathur, @.***> wrote:

Hi, any update on this because i have reinstalled Nerfstudio just now and still i get the same artifacts. And a lot of them.

Sorry, I have not gotten to it yet. I will update you when I get to it or someone else does.

— Reply to this email directly, view it on GitHub https://github.com/nerfstudio-project/nerfstudio/issues/2414#issuecomment-1787432183, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZDMZ2BGIAAFA4HFL24LB33YCEJDXAVCNFSM6AAAAAA4RWDES2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBXGQZTEMJYGM . You are receiving this because you commented.Message ID: @.***>

[yoavalon]

I got the same issue in 3.4:

In the previous version 3.3 patches have no artifacts:

[Sabisisi]

Hi, any updates on this issue? I am using a new nerfstudio version 1.0.2 and also get many artifacts on the planar projections

[haruishi43]

@tancik @THE-COB Hey, I'm the maintainer for equilib which was the dependency before your own implementation (I didn't realize it used to be a dependency until recently). From version 0.5.7 the project became licensed under Apache v2 which is not as restrictive as before, so perhaps you could use my library again for a cleaner transform. I'm sorry for the inconvenience that it caused before.