replicate
Should be easy to specify mount volume
Most images built w/ cog download most models in the setup() fn rather than being baked into the image.
And this means every time you run cog predict() you need to download the models again and again.
the two alternatives i've come to is building an app.py function that wraps the predict.py and passes all arguments into it, and using a docker run command with a mount volume specified.
so I'd like to request a field maybe named 'mount' to be added to the cog.yaml so that we don't have to do this docker run + app.py + rebuilding just to specify a mount volume.
If there's interest in a solution for this I can work on a PR 👍
In this case is the mount volume just the general source directory or some directory away from the docker context?
Going to close this for now (1 week without activity) but let me know if it should remain open.
@8W9aG a directory outside the Docker build context. A lot of these workflows like i mentioned download models in setup() to reduce the image size. But the workflow container checks if this local directory has the models, and it inevitably doesn't so it downloads the models every time cog predict is run.
the solution i've had to do is download the files into the folder it looks in, and then rebuild the image so that this directory always is populated with the models.
but i have made a number of workflows which use flux-dev fp8 model. so i now have to make an app.py file, and run that with a very big docker run command so i can make this shared model folder a mount volume for all workflows.
what would be nicer would be to simply allow for a mount volume option in the cog.yaml file. all i'd have to do then would be rebuild the image and then cog predict would only ever download the models once the first time it is ran if the models don't exist in the mount volume.
sorry didn't see your response. if u could tag me so i get a noti that would b great
@noahgsolomon Would you mind giving the following a try:
build:
fast: true
and on the CLI:
--x-local-image
I do believe this should auto detect the weights and mount them anyway (even if outside the context, e.g. via a symlink). It's a still a bit experimental but would love to hear how you get on.