TencentARC
Aligning initial input image with produced mesh
Hello! Let me first say congrats and thanks for all your work, InstantMesh is absolutely amazing!
I am wondering about the possibility of aligning the initial input image with the produced mesh, specifically extracting the camera pose of the input image which could then be applied to the mesh such that it "looks the same". I know that One-2-3-45 has this capability (code here), and one of their multi-view diffusion-created images is a recreated version of the initial image at a guessed pose. I also believe that One-2-3-45 builds their poses relative to this initial image's pose calculation.
It seems like Zero123++ (which InstantMesh uses as the multi-view diffusion model) utilizes only the content of the initial image and doesn't bother figuring out a decent "absolute" pose of the initial image, but I haven't looked too closely into the Zero123++ code quite yet.
If InstantMesh doesn't immediately have this capability, could you point me in the general direction of a solution? Use the "elevation_estimate" as seen in One-2-3-45, with 0 degree azimuth (it looks like the 6 poses produced here are assuming initial azimuth being 0 degrees?)
Thanks for any help!
Same request here :) would assume 0 degrees azimuth but not sure what the elevation actually is. I would have assumed 0 since the poses generated by zero123++ should be relative to the input image?
Same request here :) would assume 0 degrees azimuth but not sure what the elevation actually is. I would have assumed 0 since the poses generated by zero123++ should be relative to the input image?
Hello, Have you solved the problem? whats the elevation and azimuth should the input image is?
Same request here :) would assume 0 degrees azimuth but not sure what the elevation actually is. I would have assumed 0 since the poses generated by zero123++ should be relative to the input image?
Hello, Have you solved the problem? whats the elevation and azimuth should the input image is?
Sorry, I have not solved this problem. I have generally moved on from that project as well, so I probably will never need an answer. However image-to-3D object is moving so fast that there might be something better than InstantMesh already out there that has this capability? But yeah, for all future creators of related projects, it would be really nice to very easily and obviously automatically align the input image(s) with the output mesh.