verl
verl copied to clipboard
现在的代码支持actor和rollout分离吗
self.hybrid_engine = config.actor_rollout_ref.hybrid_engine
assert self.hybrid_engine, 'Currently, only support hybrid engine'
根据代码来看,目前actor, rollout, ref必须在同一个worker上。
将来会考虑支持,将actor和rollout可以分配在不同的资源和worker上吗?
Unless using asynchronous algorithms, placing actor and rollout onto different workers will only deteriorate the performance.
The reference worker can be separate. You can specify the role and initialize the reference model alone