Sheng Wang comments

Results 6 comments of


                                            Sheng Wang

Trained on small dataset with pre-trained weight, don't have good result.

update: "Didn't work, output is all cat or all dog." was trained on only 1k images. Now I train the ViT on whole dataset which have 20k images and it...

Trained on small dataset with pre-trained weight, don't have good result.

``` v = ViT( image_size = 224, patch_size = 32, num_classes = 2, dim = 512, depth = 4, heads = 8, mlp_dim = 512, dropout = 0.1, emb_dropout =...

Trained on small dataset with pre-trained weight, don't have good result.

The pertained model had a peak acc of 0.796 after 100 epochs of training. In this dataset, resnet50 can reach 90 without any modification. Is there any tuning trick I...

Trained on small dataset with pre-trained weight, don't have good result.

Yes, Ross' model (which is uploaded to timm) is used. Is pretrained model always work on small dataset?

Trained on small dataset with pre-trained weight, don't have good result.

> @JamesQFreeman ohh... well, I think I spot the error, your learning rate is way too high `1e-2`, try Karpathy's favorite LR, `3e-4` Thanks! I'll give a try.

How you collect datum from two radiologists and how to use to get consistency loss ?

In fact, there are multiple solutions: 1. duplicate the image, then you have (im1,gaze1) and (im1,gaze2) as two individual item; 2. chose one and discard the other one, which means...