Anyway to fine-tune?
I have a small subset (thousand at max) of coordinates of word bounding boxes. I don't want to train from scratch, but rather fine-tune from weights pre-trained by the icdar2015 dataset.
Is it possible, and how?
you can load the weights from the ckpt i provide, just replace the imagenet cktp with icdar ckpt
I replace the pretrained model path to ckpt you provide, but the weights under a given path disappear in the begining of training. Why? Can you give us some specific details? like python........
@argman I load the weights model.ckpt-49491 as --pretrained_model_path, but the test result is pretty bad (it cannot detect any text boxes). What is the problem here?
@mozhuangb Remember to change the path in file"checkpoint"
I want to initialize model with the trained icdar2015 model provided by author, and just set the parameter below is:
tf.app.flags.DEFINE_boolean('restore', True, 'whether to resotre from checkpoint')
and run multigpu_train.py, but error occured like below:
continue training from previous checkpoint
ERROR:tensorflow:Couldn't match files for checkpoint /tmp/east_icdar2015_resnet_v1_50_rbox/model.ckpt-49491
Traceback (most recent call last):
File "./multigpu_train.py", line 181, in <module>
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "./multigpu_train.py", line 142, in main
saver.restore(sess, ckpt)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1548, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 789, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 997, in _run
feed_dict_string, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1132, in _do_run
target_list, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InternalError: Unable to get element from the feed as bytes.
anything wrong with me??? or how to fine-tune? @argman @hiepph @mozhuangb @JmmSu @Atry thanks.
@hiepph What worked for me was the following: Instead of using the pretrained_model_path, continue training from the given checkpoint with checkpoint_path = "your_path", with restore=True. Additionally, as @JmmSu noted you have to modify the checkpoint file inside the folder so that it points to its new path.
@hiepph I have the same confusion with you, have you work out? Please tell me how can I fine-tune my own dataset? Thank you very much. Idon't understand their comments.
@YanShuang17 change the fist line in file 'checkpoint' from "/tmp/east_icdar2015_resnet_v1_50_rbox/model.ckpt-49491" to "model.ckpt-49491" and add parameter --checkpoint_path=<your_checkpoint_path> eg. --checkpoint_path=checkpoint/fine_tone/
Hi,
When I am trying to do restore=True, the error is
File "multigpu_train.py", line 180, in <module> tf.app.run() File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/platform/app.py", line 40, in run _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef) File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 299, in run _run_main(main, args) File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 250, in _run_main sys.exit(main(argv)) File "multigpu_train.py", line 141, in main saver.restore(sess, ckpt) File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/saver.py", line 1277, in restore raise ValueError("Can't load save_path when it is None.") ValueError: Can't load save_path when it is None.
I have changed the first line of checkpoint file as :
model_checkpoint_path: "model.ckpt-49491"
and passing --checkpoint_path=/path/pretrained_model_path.ckpt, but still error persists.
Can you please help with this!!!