cgraywang

Thanks for your interest @zzypeak . According to Table 3 (ablation study), the ranking model does improve the results significantly. We will also help to take a look at this...

``` parameters = model.collect_params() nlp.model.utils.multi_gpu_clip_global_norm(trainer, parameters.values(), args.clip) def _multi_gpu_clip_global_norm_scale(arrays, max_norm): """Compute the global norm'scale in order to make the 2-norm smaller than `max_norm`. """ assert len(arrays) > 0 ctx =...

Validation dataset, change batch_size smaller

bucketing example: https://github.com/dmlc/gluon-nlp/blob/973bb2d5e0b8aa369a330d33c8a3c7e3f7a302ec/scripts/nmt/train_gnmt.py#L353-L381

dataset = tf.data.TFRecordDataset(record_file).map( parser, num_parallel_calls=num_threads).shuffle(config.capacity).repeat()