TensorFlowOnSpark icon indicating copy to clipboard operation
TensorFlowOnSpark copied to clipboard

can it run use ParameterServerStrategy

Open Coder-Yifan opened this issue 3 years ago • 3 comments

I found all examples use MultiWorkerMirroredStrategy

Coder-Yifan avatar Jun 22 '22 09:06 Coder-Yifan

Yes, in theory, it should be supported, especially when using InputMode.TENSORFLOW (where Spark is really just being used as a cluster/resource manager to start up the nodes of your TF cluster).

That said, the ParameterServerStrategy has been marked experimental since the DistributionStrategies were initially introduced several years ago, so there hasn't been a lot of emphasis on it, and I would proceed with caution when using it.

leewyang avatar Jun 22 '22 21:06 leewyang

But tf.distribute.experimental.ParameterServerStrategy needs parameter “cluster_resolver”,how can i pass the parameter in main_fuc

cluster = TFCluster.run(sc, main_fun, args, args.cluster_size, num_ps=2, tensorboard=args.tensorboard,input_mode=TFCluster.InputMode.TENSORFLOW, master_node='chief')

def main_fun(args, ctx):
    """Example demonstrating loading TFRecords directly from disk (e.g. `HDFS)` without tensorflow_datasets."""
    strategy = tf.distribute.experimental.ParameterServerStrategy()#cluster_resolver```

Coder-Yifan avatar Jun 23 '22 06:06 Coder-Yifan

Pretty sure this should work, since we use TF_CONFIG to set up the cluster.

leewyang avatar Jun 23 '22 16:06 leewyang