ns-render-mesh not render at correct camera poses

[yuyingyeh]

Describe the bug ns-render-mesh command cannot render the camera poses correctly. All the renderings are at a fixed camera pose which is not related to any given camera poses. I have tested under a Windows machine with Anaconda and a Ubuntu Docker container. Similar issues are mentioned in the previous threads below, but there is no clear solution so far. I have tried --traj interpolate, spiral and filename. None of them works. https://github.com/autonomousvision/sdfstudio/issues/141#issuecomment-1617164114 https://github.com/autonomousvision/sdfstudio/issues/100#issue-1689392693

To Reproduce Steps to reproduce the behavior:

1. Follow the instructions to train and export the mesh
2. After checking the rendering results, the renderings are always under the same view, not even contain any training views

Expected behavior The rendering should be under the given camera trajectory, but now the rendering results are not.

Screenshots Please see above https://github.com/autonomousvision/sdfstudio/issues/100#issue-1689392693

[niujinshuchong]

Hi, unfortunately I don't have a windows machine to test.

Could you try to add vis.update_renderer() to this line https://github.com/autonomousvision/sdfstudio/blob/master/scripts/render_mesh.py#L140 to see if it helps? Thanks.

[yuyingyeh]

Hi,

Thanks for your fast response. I added vis.update_renderer() and re-run ns-install-cli, but the view is still incorrect. Are there anything missing for updating the code?

On the other hand, may I know your working environment? I have tried Windows and Ubuntu Docker on a linux-based cluster. I am wondering if it works under linux since this happens in the Ubuntu Docker container as well.

[niujinshuchong]

Hi, I think the main problem is open3d's rendering. Could you check if you can make it work with open3d's examples: https://github.com/isl-org/Open3D/blob/master/examples/python/visualization/headless_rendering.py? And I also find open3d also has web support for rendering (http://www.open3d.org/docs/latest/python_api/open3d.visualization.webrtc_server.html) which could be an alternative way for headless rendering.

I am using a ubuntu workstation with rtx3090 gpu.