resnet-18-tensorflow icon indicating copy to clipboard operation
resnet-18-tensorflow copied to clipboard

tensorflow.python.framework.errors_impl.OutOfRangeError: End of sequence When Using Custom Dataset

Open obong2 opened this issue 6 years ago • 0 comments

Hello, I am new and trying to train a resnet18 model with your code using my custom dataset (8 classes).

I made changes in a config file (train_scratch.sh). And I got the error below when start training (at 0 step).

Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1334, in _do_call return fn(*args) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1319, in _run_fn options, feed_dict, fetch_list, target_list, run_metadata) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.OutOfRangeError: End of sequence [[{{node IteratorGetNext}} = IteratorGetNextoutput_shapes=[[?,224,224,3], [?]], output_types=[DT_FLOAT, DT_INT32], _device="/device:CPU:0"]] [[{{node train_image/ExperimentalFunctionBufferingResourceGetNext_1}} = ExperimentalFunctionBufferingResourceGetNextoutput_types=[DT_FLOAT, DT_INT32], _device="/job:localhost/replica:0/task:0/device:GPU:1"]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "train.py", line 263, in tf.app.run() File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/platform/app.py", line 125, in run _sys.exit(main(argv)) File "train.py", line 259, in main train() File "train.py", line 233, in train feed_dict={network_train.is_train:True, network_train.lr:lr_value}) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 929, in run run_metadata_ptr) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1152, in _run feed_dict_tensor, options, run_metadata) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1328, in _do_run run_metadata) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1348, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.OutOfRangeError: End of sequence [[{{node IteratorGetNext}} = IteratorGetNextoutput_shapes=[[?,224,224,3], [?]], output_types=[DT_FLOAT, DT_INT32], _device="/device:CPU:0"]] [[node train_image/ExperimentalFunctionBufferingResourceGetNext_1 (defined at /shared/workspace/yjoh/resnet-18-tensorflow/imagenet_input.py:176) = ExperimentalFunctionBufferingResourceGetNextoutput_types=[DT_FLOAT, DT_INT 32], _device="/job:localhost/replica:0/task:0/device:GPU:1"]]

My tensorflow version is 1.12 and Python 3.6.

Let me know if more info / codes are needed. Thank you in advance!

obong2 avatar Oct 20 '19 11:10 obong2