Some issues on Insta360 equirectangular images

[LaFeuilleMorte]

Hi, thanks for open-sourcing this work. I'm using an insta360 camera to reconstruct indoor scenes. So there might be many bad conditions I may run into. For example, white walls, dynamic lighting, etc. I can reconstruct the ERP images, but when I run your cubic splitter, and fed into 3D gaussian splatting, the results looks not very good. I guess it might be that the ERP image that Insta360 captured are stitched from two back-to-back fisheye images. So there're some distortions in the ERP images that are not taken into consideration in the SphereSfm. The Sphere Cam assumes the pixels are projected onto an ideal unit cube. Could that be a reason that my reconstruction results are not good? Looking forward to your reply! PS, What if I run

colmap bundle_adjuster --BundleAdjustment.refine_principal_point=1

to further refine the splitted cubic images?

[json87]

I do not use 3DGS for the further processing. The distortion maybe one of the reasons that cause the results. Besides, I guess there are too few 3D points in the SfM sparse reconstructoin.

[mpu-creare]

@json87 awesome work. Really appreciate this implementation.

I suspect the issue is that the cubic projection swapped left/right top/down.

Here's my dense reconstruction with no changes (streetview data):

Here it is after renaming the left/right files to swap them.

Notice the denser points, especially behind that ring-sculpture.

Here's a little Python script to do the renaming:

""" This script renames the left and right views of the cubic projection to swap them """
import os

path = "sparse-cubic"
files = [os.path.join(path, f) for f in os.listdir(path)]

# Rename 2 to 2b
for f in files:
    if f.endswith('3.jpg'):
        os.rename(f, f.replace('3.jpg', '3a.jpg'))
# Rename 1 to 3
for f in files:
    if f.endswith('1.jpg'):
        os.rename(f, f.replace('1.jpg', '3.jpg'))
# Rename 3a to 1
for f in files:
    if f.endswith('3.jpg'):
        os.rename(f.replace('3.jpg', '3a.jpg'), f.replace('3.jpg', '1.jpg'))

[json87]

Thanks for your reply. I will check the code for cubic conversion.

[yanqswhu]

@json87 awesome work. Really appreciate this implementation.很棒的工作。非常感谢这个实现。

I suspect the issue is that the cubic projection swapped left/right top/down.我怀疑问题是三次投影向左/向右上/下交换。

Here's my dense reconstruction with no changes (streetview data):这是我的密集重建，没有变化（街景数据）：

Here it is after renaming the left/right files to swap them.这是在重命名左/右文件以交换它们之后。

Notice the denser points, especially behind that ring-sculpture.注意那些更密集的点，尤其是在那个环形雕塑后面。

Here's a little Python script to do the renaming:下面是一个用于重命名的小 Python 脚本：

""" This script renames the left and right views of the cubic projection to swap them """ import os

path = "sparse-cubic" files = [os.path.join(path, f) for f in os.listdir(path)]

Rename 2 to 2b

for f in files: if f.endswith('3.jpg'): os.rename(f, f.replace('3.jpg', '3a.jpg'))

Rename 1 to 3

for f in files: if f.endswith('1.jpg'): os.rename(f, f.replace('1.jpg', '3.jpg'))

Rename 3a to 1

for f in files: if f.endswith('3.jpg'): os.rename(f.replace('3.jpg', '3a.jpg'), f.replace('3.jpg', '1.jpg'))

It seems that 4.jpg and 5.jpg also need to change their names.

[Torchmm]

the bug is in this line code, rotation should be rotation.transpose() , the reason why front and back need not to rename is In this situation rotation equal to rotation.transpose() 。

[json87]

This bug has been addressed in the new commit 962daa2.