Bad mesh result..

[lms7127]

First of all, thank you for your great research :) I tried with my own 1000 images that is captured from video and removed background. But the result mesh(in refined_mesh folder) is too roughness. I want to get smooth surface mesh Is there any tips for getting good mesh?? or Did i any mistake in this way?? I'd appreciate it if you could give me some simple advice.

*I entered command like this . ( python train.py -s colmap/output -c output/gray_mtm/ -r "density" --low_poly True) *Two images below are 3DGS result and SuGaR refined_mesh result.

[Anttwo]

Hello @lms7127,

Thank you so much for your nice words!

Could I see an example of your image dataset? Looking at the Gaussian Splatting, it seems that your scene consists in a grey sweatshirt on a white background. Did you segment the object in your images, to get this white background?

1) About the background Because we were targetting full, real scene reconstruction with SuGaR, there is currently no mask management for SuGaR (I'm actually planning to add a mask functionality). Therefore, if you try to reconstruct a scene with a segmented object or a completely monochrome background, you will definitely get chaotic artifacts in the background.

This problem is not specific to SuGaR, and is common among Image-Based rendering methods. That's why your reconstruction looks messy in the background.

2) About the foreground Actually, if we take a look at the foreground object, the reconstruction is not that bad!

In general, a cloth can be difficult to reconstruct, as it can have lots of folds/creases etc. Here, the sweatshirt is almost monochrome, which is not easy to reconstruct, especially on a white background without masks.

Still, even though the surface of the sweatshirt is a bit irregular, the sleeves seems to be well reconstructed and the general shape of the mesh is correct. To further smooth the irregularities on the surface, you may try to:

• Decrease the number of vertices. Your scene basically consists in a single object with no background, so you may not need that many vertices in your mesh. Reducing the number of vertices will naturally smooth the surface. For information, when using --low_poly True, the target number of vertices is 200,000. When using train.py, you can remove low_poly True and use the argument --n_vertices_in_mesh 100_000 instead to decrease the number of vertices to 100,000. Please refer to the README.md file for more details.
• Increase smoothing during refinement. (EDITED) If you want to keep a high number of vertices but have a smoother surface, you can increase the mesh regularization factor used during refinement. You just have to increase the value of 'normal_consistency_factor': 0.1, at line 160 in train.py.
• Change the regularization loss. If the above do not work, you can try to use -r "sdf" rather than -r "density". In general, the density regularization is better for single, centered objects, but the SDF regularization is stronger. Therefore, it may produce better results for your specific scene.

I hope you will find this message helpful! Looking forward to your answer!

[lms7127]

Thank you for your response!! Yes! I recorded video and captured it. After that background is removed with white background. I'm trying to follow your advice now:) I hope the results are good!! This is my image dataset link.(https://drive.google.com/drive/folders/1Ilwtw7zWM7FXRCGEbxq7OcnYHpjLl0rn?usp=drive_link)

[MagicJoeXZ]

You just have to change the value of surface_mesh_laplacian_smoothing_factor at line 166. This will increase the surface regularization during refinement.

Should I increase or decrease the factor value to increase smoothing?

[Anttwo]

Woops, sorry, I made a small mistake in my previous message. To change the mesh regularization factor, you should not change surface_mesh_laplacian_smoothing_factor at line 166 as this is not used haha; you should change 'normal_consistency_factor': 0.1, at line 160 in train.py.

Sorry for that.