Pb with number of classed predicted

[BenLag2906]

Hi, When I started to learn less class with SSD ( 2 classes) : it diverge quickly.

Should I intiailizing my CNN differently ? beacuse my pretrained is based on 20 classes...

Or

Is there a modification to add with my softmax function ?

Could you please give a piece of advise ? regards