verl icon indicating copy to clipboard operation
verl copied to clipboard

现在的代码支持actor和rollout分离吗

Open Life-0-1 opened this issue 1 year ago • 2 comments

self.hybrid_engine = config.actor_rollout_ref.hybrid_engine assert self.hybrid_engine, 'Currently, only support hybrid engine'

根据代码来看,目前actor, rollout, ref必须在同一个worker上。

将来会考虑支持,将actor和rollout可以分配在不同的资源和worker上吗?

Life-0-1 avatar Feb 18 '25 07:02 Life-0-1

Unless using asynchronous algorithms, placing actor and rollout onto different workers will only deteriorate the performance.

vermouth1992 avatar Feb 18 '25 16:02 vermouth1992

The reference worker can be separate. You can specify the role and initialize the reference model alone

PeterSH6 avatar Feb 18 '25 16:02 PeterSH6