nerfstudio icon indicating copy to clipboard operation
nerfstudio copied to clipboard

c2w incorrect for test images in instant-ngp-data data parser?

Open hoanhle opened this issue 1 year ago • 1 comments

Describe the bug When running

ns-train nerfacto --output-dir /home/leh19/workspace/experiment/outputs/base_line --vis viewer+tensorboard  --experiment-name painting_1 --pipeline.model.background_color "random" instant-ngp-data --data /home/leh19/test_run_1/JPG/painting_1/processed 

I got the shape mismatching issue.

   assert c2w.shape == num_rays_shape + (3, 4), f"Shape mismatch: {c2w.shape} != {num_rays_shape + (3, 4)}"
AssertionError: Shape mismatch: torch.Size([4096, 4, 3]) != torch.Size([4096, 3, 4])

And this issue only happens when running with instant-ngp-data data-parser. Using nerfstudio-data data-parser was okay.

Data to reproduce: https://drive.google.com/drive/folders/1LlWcbTCr2EO9g-VhNowhW1IRiMHbLzgZ?usp=drive_link

hoanhle avatar Jul 31 '24 13:07 hoanhle

This seems weird as it worked for my data in instantNGP format. Are you sure that your data is formatted correctly (I didn't look it your data myself)? An easy test would be to see whether instantNGP takes your data. If it does, then this is indeed a bug.

Extrinsic matrices are always a pain as there are too many different standards. I would need to double-check which ones are used by Nerfstudio and instantNGP respectively.

nepfaff avatar Aug 20 '24 10:08 nepfaff