Alykhan Tejani

Add Node Selector attribute to KF plugins

5

## Why are the changes needed? Currently for flyteplugins-kftensorflow (similarly for pytorch and mpi) there are use cases where one would want to use preemptible/interuptible nodes for workers and chief...

Add Node Selector attribute to KF plugins

1

## Why are the changes needed? Currently for flyteplugins-kftensorflow (similarly for pytorch and mpi) there are use cases where one would want to use preemptible/interuptible nodes for workers and chief...

### Issue type Performance ### Have you reproduced the bug with TensorFlow Nightly? No ### Source source ### TensorFlow version v2.15.0-11-g63f5a65c7cd 2.15.1 ### Custom code No ### OS platform and...

Does HvdAllToAllEmbedding shard embeddings across ranks? Or is it copied on all ranks?

3

hi, I'm using ``` self.kv_creator = de.CuckooHashTableCreator( saver=de.FileSystemSaver(proc_size=1, proc_rank=hvd.local_rank()) ) self.emb = de.keras.layers.HvdAllToAllEmbedding(..., devices=['CPU']) ``` And I notice in your [keras callback](https://github.com/tensorflow/recommenders-addons/blob/master/tensorflow_recommenders_addons/dynamic_embedding/python/keras/callbacks.py#L71) for horovod you remove the `TrainableWrapper` and `DEResourceVariables`...

Slack channel / Discord for discussion

3

Hi, Is there / is there a possibility for a slack/discord for discussion about using this lib?

ShadowVariable Post applying gradients gives NaN value for embedding

2

**System information** - OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Debian - TensorFlow version and how it was installed (source or binary): 2.15.1 - TensorFlow-Recommenders-Addons version and how it...

How can I remove Horovod ops from the savedModel to use with TF Serving?

3

Title basically says it, I have trained a model using HorovodAllToAllEmbeddings and saved by doingg: ``` de.keras.models.de_save_model( model, export_dir, overwrite=True, include_optimizer=False, options=save_options, signatures={ "serving_default": model.get_tf_examples_serving_signature( feature_config, integrate_transform_layer=True, rename_serving_feature_names=rename_serving_feature_names, ), "serving_common_feature":...

How to save and restore a model with dynamic embeddings?

4

Hi, I am training a model with dynamic embeddings (specifically HvdAllToAllEmbeddings). I am saving the model to disk with `de.keras.models.de_save_model` and I see that it appears my dynamic embedding variables...