Donating System Memory?

[NavodPeiris]

is there a way to donate System memory instead of GPU VRAM?

this may become more economical

[KMouratidis]

I think if you don't have a GPU it will default to CPU, but you need to set --num_blocks manually:

https://github.com/bigscience-workshop/petals/blob/68585864aeae25e6398a9bb8eca65b80108fbf4e/src/petals/server/server.py#L275-L279

Might need more configuration though.