auroua
tran.tfrecords not found error
Hi, sir When run "train_nets.py" , I get the following errors:
2018-06-11 16:48:15.835525: W tensorflow/core/framework/allocator.cc:101] Allocation of 174415872 exceeds 10% of system memory. 2018-06-11 16:48:15.836087: W tensorflow/core/framework/allocator.cc:101] Allocation of 174415872 exceeds 10% of system memory. 2018-06-11 16:48:24.161151: W tensorflow/core/framework/allocator.cc:101] Allocation of 174415872 exceeds 10% of system memory. Traceback (most recent call last): File "/home/jimmy/anaconda3/envs/mlwork/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1322, in _do_call return fn(*args) File "/home/jimmy/anaconda3/envs/mlwork/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1307, in _run_fn options, feed_dict, fetch_list, target_list, run_metadata) File "/home/jimmy/anaconda3/envs/mlwork/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1409, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.NotFoundError: ./datasets/tfrecords/tran.tfrecords; No such file or directory [[Node: IteratorGetNext = IteratorGetNextoutput_shapes=[[?,112,112,3], [?]], output_types=[DT_FLOAT, DT_INT64], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "train_nets.py", line 167, in
Caused by op 'IteratorGetNext', defined at:
File "train_nets.py", line 65, in
NotFoundError (see above for traceback): ./datasets/tfrecords/tran.tfrecords; No such file or directory [[Node: IteratorGetNext = IteratorGetNextoutput_shapes=[[?,112,112,3], [?]], output_types=[DT_FLOAT, DT_INT64], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
Where I can get "tran.tfrecords" ? Thank you. Jimmy
Read the function named mx2tfrecords at ./data/mx2tfrecords/, and 'main' part in the same file shows its invoking process. You need to download the origin dataset (see: https://github.com/deepinsight/insightface), then try to use function mx2tfrecords to convert train.rec and train.idx to tran.tfrecords.
@jimmy3456 Hello, I have encountered the same problem. Can you tell me how to solve it? thank you very much.
hi WJSAI
As David said, you have to download the origin dataset first then modify the main part of "mx2tfrecords.py" at ./data/mx2tfrecords/ Here is my modify part at mx2tfrecords.py : if name == 'main': # # define parameters id2range = {} data_shape = (3, 112, 112) args = parse_args() imgrec = mx.recordio.MXIndexedRecordIO(args.idx_path, args.bin_path, 'r') s = imgrec.read_idx(0) header, _ = mx.recordio.unpack(s) print(header.label) imgidx = list(range(1, int(header.label[0]))) seq_identity = range(int(header.label[0]), int(header.label[1])) for identity in seq_identity: s = imgrec.read_idx(identity) header, _ = mx.recordio.unpack(s) a, b = int(header.label[0]), int(header.label[1]) id2range[identity] = (a, b) print('id2range', len(id2range))
# # generate tfrecords
mx2tfrecords(imgidx, imgrec, args)
#config = tf.ConfigProto(allow_soft_placement=True)
#sess = tf.Session(config=config)
# training datasets api config
#tfrecords_f = os.path.join(args.tfrecords_file_path, 'tran.tfrecords')
#dataset = tf.data.TFRecordDataset(tfrecords_f)
#dataset = dataset.map(parse_function)
#dataset = dataset.shuffle(buffer_size=30000)
#dataset = dataset.batch(32)
#iterator = dataset.make_initializable_iterator()
#next_element = iterator.get_next()
# begin iteration
#for i in range(1000):
# sess.run(iterator.initializer)
# while True:
# try:
# images, labels = sess.run(next_element)
# cv2.imshow('test', images[1, ...])
# cv2.waitKey(0)
# except tf.errors.OutOfRangeError:
# print("End of dataset")
hope this would help you
Hi David when i run the following code import tensorflow as tf image=tf.image.decode_jpeg(tf.read_file("images/test.jpg")) sess=tf.Session() print(sess.run(image))
then this error show
please tell me that how i fix it
@muneebullahkhan I guess you are using ipython. Maybe you need to check whether the path 'image/test.jpg' is consistent with your workdir. At least， I don't think there is any mistake in your code.
Hi @david-di
when i try to use function mx2tfrecords to convert train.rec and train.idx to tran.tfrecords like jimmy @jimmy3456
then this error show:
can you tell me that how i fix it? Any help will be grateful! thanks!
Thanks for the advice. And could you tell me more details about the “modifiy" . Is the code in the top modified and is the code in the down commented? Or simply give the complete code. Thanks a lot.
Hi @david-di the error below occurred.
Here is the code:
i find head.label is a 1*2 array ,so i take the first number as the label(i guess,) ,then the error will disappear ,but when i run train_nets.py it will stuck at Shuffle buffer filled
