PART icon indicating copy to clipboard operation
PART copied to clipboard

error finding in training

Open Aoshika123 opened this issue 2 years ago • 3 comments

Hello, thank you for your work. I modified the batchsize to 12 before training, and then error finding occurred after a period of time. Did the author encounter this problem before? Is it because of the lr setting problem?

Aoshika123 avatar Feb 24 '23 07:02 Aoshika123

Hi, I have tested it on several different machines and found it works well. Besides the commonly occurring Nan training in other codes, one possible problem here might be the abnormal values in the object part discovery. Slightly modifying the learning rate or batch size may help this issue.

If these do not help, modifying the hyper-parameters of part discovery (part number and queue number) may help.

zhao1f avatar Feb 24 '23 08:02 zhao1f

Did you solve the problem? I also met the same problem.

Carinazhao22 avatar Aug 02 '23 10:08 Carinazhao22

Hi, modifying the hyper-parameters of part discovery (part number and queue number) may help.

zhao1f avatar Sep 30 '23 03:09 zhao1f