petals icon indicating copy to clipboard operation
petals copied to clipboard

Direct server-to-server communication during finetuning

Open dvmazur opened this issue 1 year ago • 0 comments

This PR is meant to implement direct server-to-server communication via push messages, similar to ones in rpc_inference.

Note 2 self: minimal testing scenario

Run a server

python -m petals.cli.run_server Maykeye/TinyLLama-v0 --num_blocks 8 --new_swarm --identity_path server1.id --host_maddrs /ip4/127.0.0.1/tcp/1337

And then open ./examples/workbench_call_rpc_directly.ipynb

and replace INITIAL_PEERS and peer_id_string

dvmazur avatar Mar 09 '24 18:03 dvmazur