iterative
Importing models saved with pytorch
Using the latest release 0.2.7, I tried to import models that have been saved using pytorch like so mlem import ckpt/best.pt ckpt/best which produced the following error:
❌ Unexpected error: A load persistent id instruction was encountered,
but no persistent_load function was specified.
Please report it here: <https://github.com/iterative/mlem/issues>
From my understanding, MLEM doesn't support the new way of pytorch to pickle its model using the torch.save method (https://pytorch.org/docs/stable/generated/torch.save.html). Most likely, when we read their note:
The 1.6 release of PyTorch switched torch.save to use a new zipfile-based file format. torch.load still retains the ability to load files in the old format. If for any reason you want torch.save to use the old format, pass the kwarg _use_new_zipfile_serialization=False.
Additionally, reading at MLEM's source, found out it is "only" able to import pickled models when looking at the subclasses of the ExtImportHook (model-wise: PickleImportHook) (https://github.com/iterative/mlem/blob/HEAD/mlem/core/import_objects.py#L56-L67).
Therefore, I guess I better change the source code and use MLEM's save method, to persist models; but believe it could be a great new feature of the import command :)
Leaving it here to the pros! 👨🏼🎤
Best,
