Keep model in memory

[vsraptor]

Is there a way to load the model in memory i.e. when you run a script you just connect to the model instead of loading it every time.

This way you can speed up the startup time !

[wanoz]

Good question, let's look into it!

[wanoz]

Hey, just want to follow up on this question. Not sure if this is what you're after: we can load in the fine tuned model using: ModelPack(model='./results/', source='local')

[vsraptor]

no I meant...something like:

ModelPack(model='gpt2', source='in-mem-connect')

[wanoz]

I see... happy to chat further to discuss how to make this work

[vsraptor]

I think it will be hard... An idea is to have a Server that runs the model and expose let say REST API (.predict(), .embedings() ...) ... or socket...

I found easier way .... just load the model in interactive ipython session and use autoreload for the code i'm testing... this way model is in memory until i quit the session.. and I can change the code w/o reloading the model.