Running the transformer model with Tensor2Tensor using Mesh-Tensorflow(GPU implementation)

[Raviteja1996]

I am trying to run the transformer model with Tensor2tensor using mesh-tensorflow (GPU-implementation) but I am facing few errors.

steps to reproduce:

PROBLEM=translate_enfr_wmt32k MODEL=mtf_transformer HPARAMS=mtf_transformer_paper_tr_0_mesh_8 DATA_DIR=$HOME/t2t_data TMP_DIR=/tmp/t2t_datagen TRAIN_DIR=$HOME/t2t_train/$PROBLEM/$MODEL-$HPARAMS mkdir -p $DATA_DIR $TMP_DIR $TRAIN_DIR datagen: t2t-datagen --data_dir=$DATA_DIR --tmp_dir=$TMP_DIR --problem=$PROBLEM train: t2t-trainer --data_dir=$DATA_DIR --problem=$PROBLEM --model=$MODEL --hparams_set=$HPARAMS --output_dir=$TRAIN_DIR --train_steps=10

error tf_session.ExtendSession(self._session) tensorflow.python.framework.errors_impl.InvalidArgumentError: Multiple OpKernel registrations match NodeDef '{{node transformer/dropout/binary_op/parallel_0_1/Less}}': 'op: "Less" device_type: "CPU" constraint { name: "T" allowed_values { list { type: DT_BFLOAT16 } } }' and 'op: "Less" device_type: "CPU" constraint { name: "T" allowed_values { list { type: DT_BFLOAT16 } } }' [[transformer/dropout/binary_op/parallel_0_1/Less]]