yfeng95
Failure cases and cropping
Hi
I've been using DECA to auto-label VGGFace (so far) because I want to train a smaller pure head pose estimator on it.
Thanks a lot for publishing your code and your model. It's pretty awesome work I think.
But I noticed that failures are relatively frequent. Here is an example:
It's actually from AffectNet. The failures all look similar, in so far as a front facing slightly distorted face is predicted.
Counting them manually I found them to occur on roughly 2-5% of the images.
Secondly, I found DECA's results are really sensitive to the cropping region. Here is another example illustrating that.
Not using automatic cropping:
Normal usage:
It's documented so that's fair enough I guess. But I wonder if DECA could be made more robust with stronger scale and translation augmentation.
(Does there happen to be somebody wanting to try to use the photometric fitting code with "DECA initialization" to make accurate data sets of FLAME fits?)
Anyway, thought I should make a report.
Cheers Michael
Nevermind! My branch was outdated. I think your last patch with the scale factor fixed most of my problems.
P.S. In my own auto-labeling script I was using the MTCNN detector because it is much faster than FAN and consumes less memory. But I also got the failures I described above.
Now I switched back to FAN, and implemented a fallback to cpu for when it runs my gpu out of memory which is very rare but I have to deal with it. With these measures I get very decent results.
Thanks again for your great work :+1:
Have you fixed the failure cases that predicting distorted faces? I meet the similar problem and do not know how to fix it. I just using the pretrained weights provided by the authors. Thanks for your great help : )
I use the pretrained weights, too. As I wrote, using the most recent master branch worked for me. The DECA network does not like the way MTCNN picks the bounding boxes. FAN gives me much better results. Sometimes the DECA networks still messes up. But it's rarer than before the scaling fix. I didn't count the failures tbh. Must be in the order of magnitude of 1 in 100 or less.
Thanks! I just want to fix the 1% failure cases but do not know how to do it.
What should I do when I train with such a result? And it happens very frequently.
