Do huge batch size necessary?

[Jasonbaby]

Hi, I have tried to retrained the model with batch size 50, But it seems the model do not converge. I am wondering if the batch size is needed to be huge (more than 100)?

[TreB1eN]

I don't think so, maybe you need to tune your lr if you change the batchsize

[TreB1eN]

and about convergence, it will not converge at all after several epoches

[Jasonbaby]

Thanks a lot. Actually, I have trained 3 epoches, with emore dataset. And the loss does not decrease at all. Is it ok? So I continue my training? I had changed some codes, but not changed the Arcface head.

[Jasonbaby]

I changed the backbone model, and the re-train the model with emore dataset.

[TreB1eN]

that's not OK, the loss should start to decrease at very early stage

[Jasonbaby]

Thanks a lot. I will recheck my code.

[yxchng]

@Jasonbaby Have u solved the problem? I am also training with small batch size 64 with r34 backbone and it is not converging.

[joe-zxh]

I think the batch size need to be large, since there are a lot of batch normalization in mobileFaceNet.