Problems that arise during training

[TengfeiZeng]

Hello, I encountered such problems when I was training. At the beginning of the training, the previous several epochs can be very good to complete the training, but after training a dozen epochs, there will always be data set loading error or index out of scope such an error, do not know what is the reason。 The error is shown below，sometimes mistake number one, sometimes mistake number two：

1. File "/2DPASS-main/network/arch_2dpass.py", line 67, in p2img_mapping img_feat.append(pts_fea[batch_idx == b][p2img_idx[b]]) IndexError: list index out of range

2. File "/home/ubuntu/anaconda3/envs/2dpass/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1225, in _process_data data.reraise()
   File "/home/ubuntu/anaconda3/envs/2dpass/lib/python3.7/site-packages/torch/_utils.py", line 429, in reraise raise self.exc_type(msg) KeyError: Caught KeyError in DataLoader worker process 6.

File "/2DPASS-main/dataloader/pc_dataset.py", line 111, in getitem annotated_data = np.vectorize(self.learning_map.getitem)(annotated_data)
File "/home/ubuntu/anaconda3/envs/2dpass/lib/python3.7/site-packages/numpy/lib/function_base.py", line 2163, in call return self._vectorize_call(func=func, args=vargs)
File "/home/ubuntu/anaconda3/envs/2dpass/lib/python3.7/site-packages/numpy/lib/function_base.py", line 2246, in _vectorize_call outputs = ufunc(*inputs) KeyError: 56

[maoyongpeng]

我也会遇到同样的问题，但是报错是CUDA上的assert 错误，index out of bounds,我很困惑。 不知道是什么原因。