Siamese-tf2 icon indicating copy to clipboard operation
Siamese-tf2 copied to clipboard

使用自己準備的資料集跑完一個epoch後就沒反應

Open glad999 opened this issue 3 years ago • 1 comments

您好: 請問我使用自己準備的梅爾頻譜圖資料集(圖片大小432x288)在訓練完一個epoch後就會沒有反應停在原地,可能是發生了甚麼狀況呢?

Number of devices: 1
Configurations:
----------------------------------------------------------------------
|                     keys |                                   values|
----------------------------------------------------------------------
|               model_path | model_data/vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5|
|              input_shape |                               [105, 105]|
|               Init_Epoch |                                        0|
|                    Epoch |                                      100|
|               batch_size |                                       64|
|                  Init_lr |                                     0.01|
|                   Min_lr |                                   0.0001|
|           optimizer_type |                                      sgd|
|                 momentum |                                      0.9|
|            lr_decay_type |                                      cos|
|              save_period |                                       10|
|                 save_dir |                                     logs|
|              num_workers |                                        1|
|                num_train |                                      900|
|                  num_val |                                      100|
----------------------------------------------------------------------

[Warning] 使用sgd优化器时,建议将训练总步长设置到30000以上。
[Warning] 本次运行的总训练数据量为900,batch_size为64,共训练100个Epoch,计算出总训练步长为1400。
[Warning] 由于总训练步长为1400,小于建议总步长30000,建议设置总世代为2143。
C:\Users\little7\anaconda3\envs\tf2-gpu\lib\site-packages\keras\optimizers\optimizer_v2\adam.py:110: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
  super(Adam, self).__init__(name, **kwargs)
C:\Users\little7\anaconda3\envs\tf2-gpu\lib\site-packages\keras\optimizers\optimizer_v2\gradient_descent.py:108: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
  super(SGD, self).__init__(name, **kwargs)
Train on 900 samples, val on 100 samples, with batch size 64.

Epoch 1: LearningRateScheduler setting learning rate to 0.001.
Epoch 1/100
 6/14 [===========>..................] - ETA: 10s - loss: 0.7094 - binary_accuracy: 0.4935WARNING:tensorflow:Callback method `on_train_batch_end` is slow compared to the batch time (batch time: 0.4824s vs `on_train_batch_end` time: 0.6985s). Check your callbacks.
14/14 [==============================] - ETA: 0s - loss: 0.7034 - binary_accuracy: 0.4933

手動Ctrl+c終止後的訊息:

Traceback (most recent call last):
  File "c:\Users\little7\Downloads\Siamese-tf2-master\train.py", line 283, in <module>
    model.fit(
  File "C:\Users\little7\anaconda3\envs\tf2-gpu\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
    return fn(*args, **kwargs)
  File "C:\Users\little7\anaconda3\envs\tf2-gpu\lib\site-packages\keras\engine\training.py", line 1432, in fit 
    self._eval_data_handler = data_adapter.get_data_handler(
  File "C:\Users\little7\anaconda3\envs\tf2-gpu\lib\site-packages\keras\engine\data_adapter.py", line 1401, in get_data_handler
    return DataHandler(*args, **kwargs)
  File "C:\Users\little7\anaconda3\envs\tf2-gpu\lib\site-packages\keras\engine\data_adapter.py", line 1151, in __init__
    self._adapter = adapter_cls(
  File "C:\Users\little7\anaconda3\envs\tf2-gpu\lib\site-packages\keras\engine\data_adapter.py", line 926, in __init__
    super(KerasSequenceAdapter, self).__init__(
  File "C:\Users\little7\anaconda3\envs\tf2-gpu\lib\site-packages\keras\engine\data_adapter.py", line 800, in __init__
    peek, x = self._peek_and_restore(x)
  File "C:\Users\little7\anaconda3\envs\tf2-gpu\lib\site-packages\keras\engine\data_adapter.py", line 937, in _peek_and_restore
    return x[0], x
  File "c:\Users\little7\Downloads\Siamese-tf2-master\utils\dataloader.py", line 44, in __getitem__
    selected_path   = self.train_lines[self.train_labels[:] == c]
KeyboardInterrupt

glad999 avatar Aug 08 '22 14:08 glad999

验证集太少,读取数据的时候陷入死循环了

bubbliiiing avatar Aug 09 '22 14:08 bubbliiiing