SwapNet icon indicating copy to clipboard operation
SwapNet copied to clipboard

KeyError: Caught KeyError in DataLoader worker process 0.

Open ChenChao20180801 opened this issue 5 years ago • 6 comments

Hi, when i running inferrnce code,i get the error below,need your help

D:\python\files\SwapNet-master>python inference.py --checkpoint checkpoints/deep_fashion --cloth_dir ./data/deep_fashion/ --texture_dir ./data/deep_fashion/ --body_dir ./data/deep_fashion/ model None dataset None =====OPTIONS====== config_file : None comments : verbose : False display_winsize : 256 checkpoints_dir : ./checkpoints load_epoch : latest dataset : None dataset_mode : image cloth_representation : labels body_representation : rgb cloth_channels : 19 body_channels : 12 texture_channels : 3 pad : False load_size : 128 crop_size : 128 crop_bounds : None max_dataset_size : 50 batch_size : 8 shuffle_data : False num_workers : 0 gpu_id : 0 no_confirm : False interval : 1 warp_checkpoint : None texture_checkpoint : None checkpoint : checkpoints/deep_fashion body_dir : ./data/deep_fashion/ cloth_dir : ./data/deep_fashion/ texture_dir : ./data/deep_fashion/ results_dir : results skip_intermediates : False dataroot : None model : None name : is_train : False

The experiment directory 'results' already exists. Here are its contents: ['args.json', 'texture', 'warp']

Existing data will be overwritten! Are you sure you want to continue? (y/N): y

Set warp_checkpoint to checkpoints/deep_fashion\warp\latest_net_generator.pth Set texture_checkpoint to checkpoints/deep_fashion\texture\latest_net_generator.pth Running warp inference... Rebuilding warp from checkpoints/deep_fashion\warp\latest_net_generator.pth Not overriding: {'body_dir', 'cloth_dir', 'texture_dir', 'checkpoint'} initialize network with kaiming model [WarpModel] was created loading the model generator from checkpoints/deep_fashion\warp\latest_net_generator.pth ---------- Networks initialized ------------- [Network generator] Total number of parameters : 137.584 M

Creating dataset warp... cloth dir ./data/deep_fashion/ Extensions: ['.npz'] body dir ./data/deep_fashion/ dataset [WarpDataset] was created Warping cloth to match body segmentations in ./data/deep_fashion/... 0%| | 0/50 [00:00<?, ?img/s]D:\python\files\SwapNet-master\env\lib\site-packages\torch\nn\functional.py:2506: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details. "See the documentation of nn.Upsample for details.".format(mode)) D:\python\files\SwapNet-master\env\lib\site-packages\torch\nn\functional.py:2506: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details. "See the documentation of nn.Upsample for details.".format(mode)) D:\python\files\SwapNet-master\env\lib\site-packages\torch\nn\functional.py:2506: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details. "See the documentation of nn.Upsample for details.".format(mode)) D:\python\files\SwapNet-master\env\lib\site-packages\torch\nn\functional.py:2506: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details. "See the documentation of nn.Upsample for details.".format(mode)) 2%|█▋ | 1/50 [00:04<03:25, 4.20s/img 8%|██████▌ | 4/50 [00:04<02:15, 2.96 14%|███████████▍ | 7/50 [00:04<01:29, 20%|████████████████▏ | 10/50 [00:04<00 26%|█████████████████████ | 13/50 [00:0 32%|█████████████████████████▉ | 16/50 38%|██████████████████████████████▊ | 1 44%|███████████████████████████████████▋ 50%|████████████████████████████████████████▌ 56%|█████████████████████████████████████████████▎ 62%|██████████████████████████████████████████████████▏ 68%|███████████████████████████████████████████████████████ 74%|█████████████████████████████████████████████████████████ 80%|█████████████████████████████████████████████████████████ 86%|█████████████████████████████████████████████████████████ 92%|█████████████████████████████████████████████████████████ 98%|█████████████████████████████████████████████████████████ 100%|█████████████████████████████████████████████████████████ ████████████████████████| 50/50 [00:06<00:00, 7.49img/s] Warp results stored in results\warp Running texture inference... Rebuilding texture from checkpoints/deep_fashion\texture\latest_net_generator.pth Not overriding: {'body_dir', 'cloth_dir', 'texture_dir', 'checkpoint'} initialize network with kaiming model [TextureModel] was created loading the model generator from checkpoints/deep_fashion\texture\latest_net_generator.pth ---------- Networks initialized ------------- [Network generator] Total number of parameters : 41.900 M

Creating dataset texture... dataset [TextureDataset] was created Texturing cloth segmentations in results\warp... 0%| | 0/50 [00:00<?, ?img/s] Traceback (most recent call last): File "inference.py", line 222, in _run_texture() File "inference.py", line 184, in _run_texture _run_test_loop(texture_model, texture_dataset, webpage) File "inference.py", line 109, in run_test_loop for i, data in enumerate(dataset): File "D:\python\files\SwapNet-master\datasets_init.py", line 82, in iter for i, data in enumerate(self.dataloader): File "D:\python\files\SwapNet-master\env\lib\site-packages\torch\utils\data\dataloader.py", line 345, in next data = self._next_data() File "D:\python\files\SwapNet-master\env\lib\site-packages\torch\utils\data\dataloader.py", line 856, in _next_data return self._process_data(data) File "D:\python\files\SwapNet-master\env\lib\site-packages\torch\utils\data\dataloader.py", line 881, in _process_data data.reraise() File "D:\python\files\SwapNet-master\env\lib\site-packages\torch_utils.py", line 394, in reraise raise self.exc_type(msg) KeyError: Caught KeyError in DataLoader worker process 0. Original Traceback (most recent call last): File "D:\python\files\SwapNet-master\env\lib\site-packages\pandas\core\indexes\base.py", line 2657, in get_loc return self._engine.get_loc(key) File "pandas_libs\index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc File "pandas_libs\index.pyx", line 127, in pandas._libs.index.IndexEngine.get_loc File "pandas_libs\index.pyx", line 147, in pandas._libs.index.IndexEngine._get_loc_duplicates KeyError: './data/deep_fashion/body\MEN\Denim\id_00000080\01_1_front'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "D:\python\files\SwapNet-master\env\lib\site-packages\torch\utils\data_utils\worker.py", line 178, in _worker_loop data = fetcher.fetch(index) File "D:\python\files\SwapNet-master\env\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "D:\python\files\SwapNet-master\env\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "D:\python\files\SwapNet-master\datasets\texture_dataset.py", line 118, in getitem rois = np.rint(self.rois_df.loc[file_id].values * scale) File "D:\python\files\SwapNet-master\env\lib\site-packages\pandas\core\indexing.py", line 1500, in getitem return self._getitem_axis(maybe_callable, axis=axis) File "D:\python\files\SwapNet-master\env\lib\site-packages\pandas\core\indexing.py", line 1913, in _getitem_axis return self._get_label(key, axis=axis) File "D:\python\files\SwapNet-master\env\lib\site-packages\pandas\core\indexing.py", line 141, in _get_label return self.obj._xs(label, axis=axis) File "D:\python\files\SwapNet-master\env\lib\site-packages\pandas\core\generic.py", line 3585, in xs loc = self.index.get_loc(key) File "D:\python\files\SwapNet-master\env\lib\site-packages\pandas\core\indexes\base.py", line 2659, in get_loc return self._engine.get_loc(self._maybe_cast_indexer(key)) File "pandas_libs\index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc File "pandas_libs\index.pyx", line 127, in pandas._libs.index.IndexEngine.get_loc File "pandas_libs\index.pyx", line 147, in pandas._libs.index.IndexEngine._get_loc_duplicates KeyError: './data/deep_fashion/body\MEN\Denim\id_00000080\01_1_front'

ChenChao20180801 avatar Sep 03 '20 03:09 ChenChao20180801

Hi, I was facing the same problem. Are you running on Windows? Please check the forward and backward slashes in the command that you are running in windows. If it is still not resolved, let me know so I can detail you.

HardiRathod avatar Sep 04 '20 10:09 HardiRathod

Hi @ChenChao20180801 @HardiRathod @andrewjong im facing the same issue. Kindly help. Is quite urgent.

thanks

AyeshaShafique avatar Dec 22 '20 11:12 AyeshaShafique

Also having problem with this on windows.

Tomas1337 avatar Dec 30 '20 14:12 Tomas1337

Just reporting my findings for now just in case someone knows how to solve this.

This problem occurs when enumerating on the self.dataloader on the texture stage. It has no problem on the warp stage. Inspecting self.dataloader on Warp stage

special variables
function variables
batch_sampler:<torch.utils.data.sampler.BatchSampler object at 0x00000202630D0648>
batch_size:1
dataset:<datasets.warp_dataset.WarpDataset object at 0x0000020257C15548>
drop_last:False
multiprocessing_context:None
num_workers:4
pin_memory:False
sampler:<torch.utils.data.sampler.RandomSampler object at 0x0000020257C154C8>
timeout:0
worker_init_fn:None
_DataLoader__initialized:True
_DataLoader__multiprocessing_context:None
_auto_collation:True
_dataset_kind:0
_index_sampler:<torch.utils.data.sampler.BatchSampler object at 0x00000202630D0648>

while running the texture script:

special variables
function variables
batch_sampler:<torch.utils.data.sampler.BatchSampler object at 0x00000202A5145A08>
batch_size:1
dataset:<datasets.texture_dataset.TextureDataset object at 0x00000202B58BD488>
drop_last:False
multiprocessing_context:None
num_workers:4
pin_memory:False
sampler:<torch.utils.data.sampler.RandomSampler object at 0x00000202B56D6DC8>
timeout:0
worker_init_fn:None
_DataLoader__initialized:True
_DataLoader__multiprocessing_context:None
_auto_collation:True
_dataset_kind:0
_index_sampler:<torch.utils.data.sampler.BatchSampler object at 0x00000202A5145A08>

Tomas1337 avatar Dec 30 '20 15:12 Tomas1337

Hi, I was facing the same problem. Are you running on Windows? Please check the forward and backward slashes in the command that you are running in windows. If it is still not resolved, let me know so I can detail you.

@HardiRathod

Backslashes seem to be consisent after some modifications. I suspect that its the slashes somewhere when creating the dataset for the Texture portion. I just cant trace/debug it

Tomas1337 avatar Dec 30 '20 18:12 Tomas1337

same issue. Any solution yet?

heyJatin avatar Jun 21 '22 13:06 heyJatin