zsnoob

accelerate sim_matrix process in multi-GPU

1

I edit two main things: 1. Deleting the "loss.mean()" that do nothing. DDP provides automatically gradient synchronization. 2. Refer to this comment, https://github.com/openai/CLIP/issues/132#issuecomment-908004353 we will do every similarity calculation locally....

In main_task_retrieval.py, fuction "train_epoch", we can see: ``` python if n_gpu > 1: loss = loss.mean() # mean() to average on multi-gpu. ``` But in modeling.py, there is: ``` python...