dmlc
How to implement the 'Bounded Delay' consistency model in ps-lite?
'Bounded Delay' consistency model is mentioned in the paper "Scaling Distributed Machine Learning with the Parameter Server". How to implement it in ps-lite? Can give some tips? Thx.
there is an example,
t1 = push(..)
t2 = push(..)
wait(t1)
t3 = push(...)
wait(t2)
at this point, t1 and t2 are guaranteed to be finished.
@mli thx, I want to implement it in MXNet. (1) if I implement it as you say, which file I should modify? kvstore_dist.h? (2) if implement it in kvstore server not in worker can be better? if I implement it in server which file I should modify? kv_app.h?
I think the 'Bounded Delay' consistency model means that waiting for all the previous \tau updates (pushes) of all the workers have been applied. It seems that the wait function only waits for the local push to be done? Or, the timestamp returned by the push function is a global timestamp, so that the wait function is actually something like a global barrier?
As @xcgoner said, the code above is not a SSP algorithm.
Local wait only guarantees memory that received parameters be available .
