Anttwo
Sharing some important tricks and tips
@Anttwo Hello Antoine, I hope you don't mind me posting a little bit of a tutorial on how to set up and get their best data set of pictures. This is the only place I know where to put it.
Here is a picture of a simple test I did this morning to show everyone how easy it is when you prepare the images properly. This is specifically for scanning objects, but much of what I will share will help with the scanning of environments.
First, @chenin_wang I took some of your photos to create a much better result. It is not perfect, but I will explain why.
This was the original scan done with over 1000 photos.
And this is the one I did with just 86.
My test was still not great, but definitely much better. The reason I could not do a better job was because of the extremely low resolution. This resolution removes much of the details needed for the programs to determine the 3D volume.
So first, use higher resolution.
Second, sharpen the image! Use some photo editing software to do this. I use Topaz Sharpen to do this. (I don't work for them, I just like the product).
The third step which is the most important for a good result is masking the pictures with a transparent background. I use rembg from github. You can do batch masking as well using "rembg p path/to/input path/to/output".
Here are the examples.
The original:
And the masked:
It needs to be transparent. Just using photo software to make the background all black or all white wont work.
Here is another example:
The original.
The masked:
And the result:
As a note, I used about 120 photos for this chair scan. I also used all the shortest and quickest setting in SuGaR but still get these kinds of results.
This is the type of lazy susan I use to turn my larger objects. With the masking program rembg all I have to do is have my background a solid different color. If I shoot something dark, I use white. If I shoot something light, I use black. Try not to use green. I worked in the film industry for years and the green causes all sorts of problems like any other color. What it does is it reflect the color off the subject and is noticible in the final output.
And lastly make sure you have no shadows. If you use lights have them pointing at the object from where the camera is. If you put lights on the sides, it will create shadows and your output will have dark patches all over. I hope this helps everyone out.
Best,
Kit
Ps. Antoine, if you are reading this, Please, Please make the windows conversion! I can do this now, but I have so many steps and I know what you are doing here will make a massive impact in the film industry. I know I will be using this myself for the movies I am producing. I will even put your name in the movie credits! :-)
@kitmallet Thank you so much! All those who have achieved good results will appreciate your sharing!
Hello @kitmallet,
Awesome, this is a great idea, thank you so much for this! This is indeed super useful.
I was planning to do a Tips&Tricks section as well, but I've been very busy recently with different side projects, so I did not find time to do it, sorry for that. I will definitely add your Tips&Tricks to the github page and/or the webpage (and cite your name, of course)!
Concerning masks: The Vanilla Gaussian Splatting code handles precomputed masks but not SuGaR, I'm planning to add this functionality to SuGaR as well so that you don't end up with black or white floaters in the scene when using masks.
About Windows: @kitmallet You're right about Windows, I wrote this in my to-do list but I haven't done it yet. The changes are very simple to do, so I have no excuse! I'm gonna update the code as soon as possible. Meanwhile, if you like to run research code available on GitHub (most of the time, tutorials are written for Linux), you can install WSL2 (Windows Subsystem for Linux v2). WSL2 is a very lightweight and super efficient virtual machine made by Microsoft for Windows. It is super easy to install and run, and it allows you to have a version of Ubuntu running on your computer that can communicate in real time with all your Windows folders. With WSL2, you can follow any tutorial written for Linux, and it will work. No need to perform a dual-booting or anything, you just have to open a specific terminal in your Windows OS. Actually, I'm using Windows with WSL2 on my personal laptop, and it works like a charm!
Sharing results
Also, I've seen many cool results by you guys in the GitHub issues, so I'm thinking about making a "Gallery" on the webpage, in which I would add images of reconstructions made by different users (along with the user's names, of course!).
What do you think?
@Anttwo A gallery is a great idea. It would also be nice for people to put in their system that they are using (GPU etc), and what settings they used. I am curious about this as well. It is nice to know the limitations. I only have an RTX 3060 (12gb) but total memory I have 72gb with the shared GPU memory. Very limited, but getting good results with smaller data sets.
@kitmallet Hi, thanks for your share. But I run into a problem, after I matting the image, the convert.py only reconstructed with 2 images. Do you have any idea why is this?
Hello @ShengL333,
This is a well-known issue with COLMAP.
Please refer to the point 2. Estimate camera poses with COLMAP of the Tips section in the README.md file.
You will find a very simple solution to your problem!
Wish you a nice day!
Hello @ShengL333,
This is a well-known issue with COLMAP. Please refer to the point 2. Estimate camera poses with COLMAP of the Tips section in the
README.mdfile. You will find a very simple solution to your problem!Wish you a nice day!
Thank you so much!
Hi @kitmallet, would it be possible to share your dataset of the woman on the chair? Thank you!
