models icon indicating copy to clipboard operation
models copied to clipboard

Resnet50, SqueezeNet and VGG19 represented twice from different source frameworks

Open crahrig opened this issue 7 years ago • 4 comments

The model zoo currently contains three models (Resnet50, SqueezeNet and VGG19) that each have two variants which is confusing to end consumers.

Ideally these should be de-duplicated. If that doesn't make sense then they should state their differences outside of origin framework and be organized in a way that places them all in the same sub-folder/path on the repo.

Following are pointers to the existing duplicated models:

  • Resnet50
    • https://github.com/onnx/models/tree/master/resnet50
    • https://s3.amazonaws.com/onnxmodelzoo/resnet/resnet50v1/resnet50v1.onnx
  • SqueezeNet (different revisions of the network, likely makes sense to leave both but should consolidate their paths)
    • https://github.com/onnx/models/tree/master/squeezenet
    • https://github.com/onnx/models/tree/master/models/image_classification/squeezenet
  • VGG19
    • https://github.com/onnx/models/tree/master/vgg19
    • https://s3.amazonaws.com/onnxmodelzoo/vgg/vgg16/vgg16.onnx

crahrig avatar Jul 24 '18 17:07 crahrig

Thanks for the note @crahrig ! Fair point. Labeled and you are welcomed to submit a PR to merge into models listed here where there is more detailed documentation.

lupesko avatar Jul 26 '18 06:07 lupesko

For the case of SqueezeNet models, there are not only two different models, but also one of the models results in poor accuracy values when I execute it.

For the first SqueezeNet model the link is given as the following: https://github.com/onnx/models/tree/master/models/image_classification/squeezenet

The probabilities that I get after executing this model is given below:

class=n02123045 tabby, tabby cat ; probability=0.411851
class=n02124075 Egyptian cat ; probability=0.295258
class=n02123159 tiger cat ; probability=0.251817
class=n02127052 lynx, catamount ; probability=0.037665
class=n02114367 timber wolf, grey wolf, gray wolf, Canis lupus ; probability=0.001155

Whereas, for the second model, the probabilities predicted for each class is pretty inaccurate. The link to this model is given as the following: https://github.com/onnx/models/tree/master/squeezenet

The probabilities are also given below:

class=n15075141 toilet tissue, toilet paper, bathroom tissue ; probability=1.000000
class=n02319095 sea urchin ; probability=1.000000
class=n02395406 hog, pig, grunter, squealer, Sus scrofa ; probability=1.000000
class=n02391049 zebra ; probability=1.000000
class=n02389026 sorrel ; probability=1.000000

idilsulo avatar Nov 15 '18 18:11 idilsulo

Right now, I think, the top priority thing is getting them correct. All models here should conform to ONNX standard, and have a reasonable accuracy. I don't think it's a good time point to dedup models. Let's close issues like: #90 first.

snnn avatar Nov 15 '18 18:11 snnn

@snnn What I mentioned above is not related to models being duplicate but rather one model having a really low accuracy value (not being correct) compared to the other one.

idilsulo avatar Nov 17 '18 08:11 idilsulo