tensorflow
Tensorflow models ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8 Model training for object detection
Hi everyone I create an application that recognizes the names of my cats. I would like to use the model "ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8" and train it using my data and tensorflow 2.3.0
Question 1. How to configure pipeline.config file What value should I give for "fine_tune_checkpoint"? After unpacking the model in the checkpoint directory I have 3 files: checkpoint, ckpt-0.index, ckpt-0.data-00000-of-00001 Should I indicate any of these files or their directory?
Question 2. Num_classes: 90. If my dataset has 4 classes, should num_classes be 94 or 4?
@Pooya-Mohammadi-K
Q1. In the pipeline.config file I set: fine_tune_checkpoint: "/home/dominik/Downloads/repo/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/checkpoint/ckpt-0"
after starting the model_main_tf2.py script, the system returns the message: tensorflow.python.framework.errors_impl.FailedPreconditionError: /home/dominik/Downloads/repo/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/checkpoint.tmpb8d6f3e0407948dfa546755fd5e7eb78; Is a directory
python3 model_main_tf2.py --model_dir=/home/dominik/Downloads/repo/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/ --num_train_steps=10000 --pipeline_config_path=/home/dominik/Downloads/repo/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/pipeline.config --alsologtostderr 2020-09-22 19:51:39.433603: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1 2020-09-22 19:51:41.009700: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcuda.so.1 2020-09-22 19:51:41.052996: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: pciBusID: 0000:03:00.0 name: Quadro K5200 computeCapability: 3.5 coreClock: 0.771GHz coreCount: 12 deviceMemorySize: 7.93GiB deviceMemoryBandwidth: 179.05GiB/s 2020-09-22 19:51:41.053070: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1 2020-09-22 19:51:41.053199: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcublas.so.10'; dlerror: libcublas.so.10: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-10.1/lib64 2020-09-22 19:51:41.055241: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10 2020-09-22 19:51:41.055579: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10 2020-09-22 19:51:41.057629: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10 2020-09-22 19:51:41.058701: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10 2020-09-22 19:51:41.058803: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudnn.so.7'; dlerror: libcudnn.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-10.1/lib64 2020-09-22 19:51:41.058817: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1753] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform. Skipping registering GPU devices... 2020-09-22 19:51:41.060154: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2020-09-22 19:51:41.072644: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 2394645000 Hz 2020-09-22 19:51:41.074723: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55ceb7d81ae0 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2020-09-22 19:51:41.074767: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2020-09-22 19:51:41.078687: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix: 2020-09-22 19:51:41.078724: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263] WARNING:tensorflow:There are non-GPU devices in tf.distribute.Strategy, not using nccl allreduce. W0922 19:51:41.083575 140036650096448 cross_device_ops.py:1202] There are non-GPU devices in tf.distribute.Strategy, not using nccl allreduce. INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0',) I0922 19:51:41.085505 140036650096448 mirrored_strategy.py:341] Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0',) INFO:tensorflow:Maybe overwriting train_steps: 10000 I0922 19:51:41.092179 140036650096448 config_util.py:552] Maybe overwriting train_steps: 10000 INFO:tensorflow:Maybe overwriting use_bfloat16: False I0922 19:51:41.092288 140036650096448 config_util.py:552] Maybe overwriting use_bfloat16: False WARNING:tensorflow:num_readers has been reduced to 1 to match input file shards. W0922 19:51:41.119355 140036650096448 dataset_builder.py:82] num_readers has been reduced to 1 to match input file shards. WARNING:tensorflow:From /home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/object_detection/builders/dataset_builder.py:96: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.experimental.AUTOTUNE)instead. If sloppy execution is desired, usetf.data.Options.experimental_deterministic. W0922 19:51:41.123609 140036650096448 deprecation.py:317] From /home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/object_detection/builders/dataset_builder.py:96: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.experimental.AUTOTUNE)instead. If sloppy execution is desired, usetf.data.Options.experimental_deterministic. WARNING:tensorflow:From /home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/object_detection/builders/dataset_builder.py:174: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.data.Dataset.map()
W0922 19:51:41.166138 140036650096448 deprecation.py:317] From /home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/object_detection/builders/dataset_builder.py:174: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.data.Dataset.map() WARNING:tensorflow:From /home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py:201: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version. Instructions for updating: Create a tf.sparse.SparseTensorand usetf.sparse.to_denseinstead. W0922 19:51:47.113795 140036650096448 deprecation.py:317] From /home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py:201: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version. Instructions for updating: Create atf.sparse.SparseTensorand usetf.sparse.to_denseinstead. WARNING:tensorflow:From /home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py:201: sample_distorted_bounding_box (from tensorflow.python.ops.image_ops_impl) is deprecated and will be removed in a future version. Instructions for updating:seed2arg is deprecated.Use sample_distorted_bounding_box_v2 instead. W0922 19:51:49.656738 140036650096448 deprecation.py:317] From /home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py:201: sample_distorted_bounding_box (from tensorflow.python.ops.image_ops_impl) is deprecated and will be removed in a future version. Instructions for updating:seed2arg is deprecated.Use sample_distorted_bounding_box_v2 instead. WARNING:tensorflow:From /home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/object_detection/inputs.py:259: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Usetf.castinstead. W0922 19:51:51.048091 140036650096448 deprecation.py:317] From /home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/object_detection/inputs.py:259: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Usetf.castinstead. WARNING:tensorflow:From /home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/object_detection/model_lib_v2.py:355: set_learning_phase (from tensorflow.python.keras.backend) is deprecated and will be removed after 2020-10-11. Instructions for updating: Simply pass a True/False value to thetrainingargument of thecallmethod of your layer or model. W0922 19:52:52.690051 140025320695552 deprecation.py:317] From /home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/object_detection/model_lib_v2.py:355: set_learning_phase (from tensorflow.python.keras.backend) is deprecated and will be removed after 2020-10-11. Instructions for updating: Simply pass a True/False value to thetrainingargument of thecall method of your layer or model. WARNING:tensorflow:Unresolved object in checkpoint: (root).model._groundtruth_lists W0922 19:54:23.778589 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._groundtruth_lists WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor W0922 19:54:23.779017 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor WARNING:tensorflow:Unresolved object in checkpoint: (root).model._batched_prediction_tensor_names W0922 19:54:23.779086 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._batched_prediction_tensor_names WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._box_prediction_head W0922 19:54:23.779145 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._box_prediction_head WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads W0922 19:54:23.779201 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._sorted_head_names W0922 19:54:23.779255 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._sorted_head_names WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._additional_projection_layers W0922 19:54:23.779308 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._additional_projection_layers WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads W0922 19:54:23.779361 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers W0922 19:54:23.779413 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._box_prediction_head._box_encoder_layers W0922 19:54:23.779467 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._box_prediction_head._box_encoder_layers WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background W0922 19:54:23.779536 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._additional_projection_layers.0 W0922 19:54:23.779589 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._additional_projection_layers.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._additional_projection_layers.1 W0922 19:54:23.779642 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._additional_projection_layers.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._additional_projection_layers.2 W0922 19:54:23.779693 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._additional_projection_layers.2 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._additional_projection_layers.3 W0922 19:54:23.779745 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._additional_projection_layers.3 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._additional_projection_layers.4 W0922 19:54:23.779797 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._additional_projection_layers.4 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings W0922 19:54:23.779847 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background W0922 19:54:23.779899 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.BoxPredictionTower W0922 19:54:23.779951 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.BoxPredictionTower WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.ClassPredictionTower W0922 19:54:23.780002 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.ClassPredictionTower WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._box_prediction_head._box_encoder_layers.0 W0922 19:54:23.780065 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._box_prediction_head._box_encoder_layers.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background._class_predictor_layers W0922 19:54:23.780120 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background._class_predictor_layers WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0 W0922 19:54:23.780174 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1 W0922 19:54:23.780226 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2 W0922 19:54:23.780278 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3 W0922 19:54:23.780330 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4 W0922 19:54:23.780383 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0 W0922 19:54:23.780434 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1 W0922 19:54:23.780486 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2 W0922 19:54:23.780537 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3 W0922 19:54:23.780589 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4 W0922 19:54:23.780641 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.BoxPredictionTower.0 W0922 19:54:23.780705 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.BoxPredictionTower.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.BoxPredictionTower.1 W0922 19:54:23.780759 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.BoxPredictionTower.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.BoxPredictionTower.2 W0922 19:54:23.780813 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.BoxPredictionTower.2 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.BoxPredictionTower.3 W0922 19:54:23.780878 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.BoxPredictionTower.3 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.ClassPredictionTower.0 W0922 19:54:23.780936 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.ClassPredictionTower.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.ClassPredictionTower.1 W0922 19:54:23.781007 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.ClassPredictionTower.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.ClassPredictionTower.2 W0922 19:54:23.781114 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.ClassPredictionTower.2 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.ClassPredictionTower.3 W0922 19:54:23.781209 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.ClassPredictionTower.3 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._box_prediction_head._box_encoder_layers.0.kernel W0922 19:54:23.781311 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._box_prediction_head._box_encoder_layers.0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._box_prediction_head._box_encoder_layers.0.bias W0922 19:54:23.781372 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._box_prediction_head._box_encoder_layers.0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background._class_predictor_layers.0 W0922 19:54:23.781429 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background._class_predictor_layers.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.1 W0922 19:54:23.781497 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.2 W0922 19:54:23.781552 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.2 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.4 W0922 19:54:23.781607 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.4 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.5 W0922 19:54:23.781662 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.5 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.7 W0922 19:54:23.781724 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.7 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.8 W0922 19:54:23.781776 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.8 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.10 W0922 19:54:23.781828 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.10 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.11 W0922 19:54:23.781880 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.11 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.1 W0922 19:54:23.781932 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.2 W0922 19:54:23.781983 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.2 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.4 W0922 19:54:23.782035 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.4 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.5 W0922 19:54:23.782140 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.5 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.7 W0922 19:54:23.782216 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.7 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.8 W0922 19:54:23.782301 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.8 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.10 W0922 19:54:23.782362 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.10 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.11 W0922 19:54:23.782421 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.11 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.1 W0922 19:54:23.782478 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.2 W0922 19:54:23.782557 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.2 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.4 W0922 19:54:23.782620 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.4 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.5 W0922 19:54:23.782672 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.5 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.7 W0922 19:54:23.782725 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.7 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.8 W0922 19:54:23.782779 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.8 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.10 W0922 19:54:23.782832 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.10 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.11 W0922 19:54:23.782885 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.11 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.1 W0922 19:54:23.782936 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.2 W0922 19:54:23.782988 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.2 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.4 W0922 19:54:23.783040 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.4 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.5 W0922 19:54:23.783092 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.5 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.7 W0922 19:54:23.783143 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.7 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.8 W0922 19:54:23.783195 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.8 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.10 W0922 19:54:23.783247 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.10 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.11 W0922 19:54:23.783298 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.11 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.1 W0922 19:54:23.783350 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.2 W0922 19:54:23.783402 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.2 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.4 W0922 19:54:23.783453 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.4 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.5 W0922 19:54:23.783517 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.5 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.7 W0922 19:54:23.783570 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.7 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.8 W0922 19:54:23.783622 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.8 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.10 W0922 19:54:23.783674 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.10 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.11 W0922 19:54:23.783725 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.11 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.1 W0922 19:54:23.783776 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.2 W0922 19:54:23.823598 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.2 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.4 W0922 19:54:23.823754 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.4 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.5 W0922 19:54:23.823842 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.5 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.7 W0922 19:54:23.823925 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.7 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.8 W0922 19:54:23.824003 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.8 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.10 W0922 19:54:23.824081 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.10 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.11 W0922 19:54:23.824158 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.11 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.1 W0922 19:54:23.824234 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.2 W0922 19:54:23.824309 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.2 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.4 W0922 19:54:23.824384 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.4 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.5 W0922 19:54:23.824460 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.5 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.7 W0922 19:54:23.824536 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.7 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.8 W0922 19:54:23.824611 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.8 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.10 W0922 19:54:23.824686 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.10 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.11 W0922 19:54:23.824761 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.11 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.1 W0922 19:54:23.824835 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.2 W0922 19:54:23.824909 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.2 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.4 W0922 19:54:23.824983 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.4 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.5 W0922 19:54:23.825056 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.5 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.7 W0922 19:54:23.825130 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.7 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.8 W0922 19:54:23.825204 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.8 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.10 W0922 19:54:23.825279 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.10 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.11 W0922 19:54:23.825353 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.11 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.1 W0922 19:54:23.825427 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.2 W0922 19:54:23.825501 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.2 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.4 W0922 19:54:23.825575 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.4 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.5 W0922 19:54:23.825648 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.5 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.7 W0922 19:54:23.825722 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.7 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.8 W0922 19:54:23.825795 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.8 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.10 W0922 19:54:23.826811 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.10 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.11 W0922 19:54:23.826913 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.11 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.1 W0922 19:54:23.826996 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.2 W0922 19:54:23.827075 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.2 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.4 W0922 19:54:23.827152 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.4 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.5 W0922 19:54:23.827228 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.5 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.7 W0922 19:54:23.827302 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.7 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.8 W0922 19:54:23.827376 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.8 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.10 W0922 19:54:23.827450 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.10 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.11 W0922 19:54:23.827543 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.11 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.BoxPredictionTower.0.kernel W0922 19:54:23.827618 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.BoxPredictionTower.0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.BoxPredictionTower.1.kernel W0922 19:54:23.827692 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.BoxPredictionTower.1.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.BoxPredictionTower.2.kernel W0922 19:54:23.827766 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.BoxPredictionTower.2.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.BoxPredictionTower.3.kernel W0922 19:54:23.827839 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.BoxPredictionTower.3.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.ClassPredictionTower.0.kernel W0922 19:54:23.827912 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.ClassPredictionTower.0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.ClassPredictionTower.1.kernel W0922 19:54:23.827985 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.ClassPredictionTower.1.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.ClassPredictionTower.2.kernel W0922 19:54:23.828057 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.ClassPredictionTower.2.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.ClassPredictionTower.3.kernel W0922 19:54:23.828130 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._head_scope_conv_layers.ClassPredictionTower.3.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background._class_predictor_layers.0.kernel W0922 19:54:23.828207 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background._class_predictor_layers.0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background._class_predictor_layers.0.bias W0922 19:54:23.828282 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background._class_predictor_layers.0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.1.axis W0922 19:54:23.828356 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.1.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.1.gamma W0922 19:54:23.828429 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.1.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.1.beta W0922 19:54:23.828502 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.1.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.1.moving_mean W0922 19:54:23.828576 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.1.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.1.moving_variance W0922 19:54:23.828648 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.1.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.4.axis W0922 19:54:23.828720 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.4.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.4.gamma W0922 19:54:23.828793 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.4.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.4.beta W0922 19:54:23.828866 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.4.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.4.moving_mean W0922 19:54:23.828945 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.4.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.4.moving_variance W0922 19:54:23.829043 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.4.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.7.axis W0922 19:54:23.829118 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.7.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.7.gamma W0922 19:54:23.829191 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.7.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.7.beta W0922 19:54:23.829264 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.7.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.7.moving_mean W0922 19:54:23.829337 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.7.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.7.moving_variance W0922 19:54:23.829411 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.7.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.10.axis W0922 19:54:23.829483 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.10.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.10.gamma W0922 19:54:23.829556 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.10.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.10.beta W0922 19:54:23.829629 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.10.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.10.moving_mean W0922 19:54:23.829703 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.10.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.10.moving_variance W0922 19:54:23.829776 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.0.10.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.1.axis W0922 19:54:23.829849 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.1.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.1.gamma W0922 19:54:23.829922 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.1.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.1.beta W0922 19:54:23.829994 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.1.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.1.moving_mean W0922 19:54:23.830067 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.1.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.1.moving_variance W0922 19:54:23.830140 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.1.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.4.axis W0922 19:54:23.830214 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.4.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.4.gamma W0922 19:54:23.830287 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.4.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.4.beta W0922 19:54:23.830360 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.4.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.4.moving_mean W0922 19:54:23.830433 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.4.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.4.moving_variance W0922 19:54:23.830507 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.4.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.7.axis W0922 19:54:23.830580 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.7.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.7.gamma W0922 19:54:23.830653 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.7.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.7.beta W0922 19:54:23.830726 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.7.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.7.moving_mean W0922 19:54:23.830801 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.7.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.7.moving_variance W0922 19:54:23.830873 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.7.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.10.axis W0922 19:54:23.830946 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.10.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.10.gamma W0922 19:54:23.831020 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.10.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.10.beta W0922 19:54:23.831093 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.10.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.10.moving_mean W0922 19:54:23.831166 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.10.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.10.moving_variance W0922 19:54:23.831240 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.1.10.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.1.axis W0922 19:54:23.831313 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.1.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.1.gamma W0922 19:54:23.831386 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.1.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.1.beta W0922 19:54:23.831460 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.1.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.1.moving_mean W0922 19:54:23.831557 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.1.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.1.moving_variance W0922 19:54:23.831632 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.1.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.4.axis W0922 19:54:23.831705 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.4.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.4.gamma W0922 19:54:23.831778 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.4.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.4.beta W0922 19:54:23.831851 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.4.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.4.moving_mean W0922 19:54:23.831923 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.4.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.4.moving_variance W0922 19:54:23.831997 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.4.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.7.axis W0922 19:54:23.879255 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.7.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.7.gamma W0922 19:54:23.879395 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.7.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.7.beta W0922 19:54:23.879519 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.7.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.7.moving_mean W0922 19:54:23.879623 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.7.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.7.moving_variance W0922 19:54:23.879722 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.7.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.10.axis W0922 19:54:23.879817 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.10.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.10.gamma W0922 19:54:23.879915 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.10.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.10.beta W0922 19:54:23.880089 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.10.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.10.moving_mean W0922 19:54:23.880271 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.10.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.10.moving_variance W0922 19:54:23.880452 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.2.10.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.1.axis W0922 19:54:23.880631 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.1.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.1.gamma W0922 19:54:23.880805 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.1.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.1.beta W0922 19:54:23.880981 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.1.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.1.moving_mean W0922 19:54:23.881139 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.1.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.1.moving_variance W0922 19:54:23.881241 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.1.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.4.axis W0922 19:54:23.881338 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.4.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.4.gamma W0922 19:54:23.881432 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.4.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.4.beta W0922 19:54:23.881526 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.4.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.4.moving_mean W0922 19:54:23.881621 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.4.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.4.moving_variance W0922 19:54:23.881715 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.4.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.7.axis W0922 19:54:23.881809 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.7.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.7.gamma W0922 19:54:23.881903 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.7.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.7.beta W0922 19:54:23.881995 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.7.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.7.moving_mean W0922 19:54:23.882089 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.7.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.7.moving_variance W0922 19:54:23.882182 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.7.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.10.axis W0922 19:54:23.884883 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.10.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.10.gamma W0922 19:54:23.885062 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.10.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.10.beta W0922 19:54:23.885257 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.10.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.10.moving_mean W0922 19:54:23.885375 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.10.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.10.moving_variance W0922 19:54:23.885477 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.3.10.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.1.axis W0922 19:54:23.885577 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.1.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.1.gamma W0922 19:54:23.885674 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.1.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.1.beta W0922 19:54:23.885771 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.1.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.1.moving_mean W0922 19:54:23.885868 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.1.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.1.moving_variance W0922 19:54:23.885967 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.1.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.4.axis W0922 19:54:23.886063 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.4.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.4.gamma W0922 19:54:23.886157 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.4.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.4.beta W0922 19:54:23.886253 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.4.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.4.moving_mean W0922 19:54:23.886348 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.4.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.4.moving_variance W0922 19:54:23.886442 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.4.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.7.axis W0922 19:54:23.886537 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.7.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.7.gamma W0922 19:54:23.886632 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.7.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.7.beta W0922 19:54:23.886726 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.7.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.7.moving_mean W0922 19:54:23.886821 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.7.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.7.moving_variance W0922 19:54:23.886915 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.7.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.10.axis W0922 19:54:23.887011 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.10.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.10.gamma W0922 19:54:23.887107 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.10.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.10.beta W0922 19:54:23.887201 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.10.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.10.moving_mean W0922 19:54:23.887295 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.10.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.10.moving_variance W0922 19:54:23.887390 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.box_encodings.4.10.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.1.axis W0922 19:54:23.887507 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.1.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.1.gamma W0922 19:54:23.887605 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.1.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.1.beta W0922 19:54:23.911468 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.1.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.1.moving_mean W0922 19:54:23.911682 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.1.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.1.moving_variance W0922 19:54:23.911842 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.1.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.4.axis W0922 19:54:23.911959 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.4.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.4.gamma W0922 19:54:23.912072 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.4.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.4.beta W0922 19:54:23.912183 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.4.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.4.moving_mean W0922 19:54:23.912292 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.4.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.4.moving_variance W0922 19:54:23.912400 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.4.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.7.axis W0922 19:54:23.912508 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.7.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.7.gamma W0922 19:54:23.912616 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.7.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.7.beta W0922 19:54:23.912725 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.7.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.7.moving_mean W0922 19:54:23.912832 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.7.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.7.moving_variance W0922 19:54:23.912939 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.7.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.10.axis W0922 19:54:23.913044 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.10.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.10.gamma W0922 19:54:23.913150 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.10.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.10.beta W0922 19:54:23.913257 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.10.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.10.moving_mean W0922 19:54:23.913363 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.10.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.10.moving_variance W0922 19:54:23.913469 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.0.10.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.1.axis W0922 19:54:23.913575 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.1.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.1.gamma W0922 19:54:23.913680 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.1.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.1.beta W0922 19:54:23.913787 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.1.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.1.moving_mean W0922 19:54:23.913894 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.1.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.1.moving_variance W0922 19:54:23.914000 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.1.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.4.axis W0922 19:54:23.914107 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.4.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.4.gamma W0922 19:54:23.914212 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.4.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.4.beta W0922 19:54:23.914318 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.4.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.4.moving_mean W0922 19:54:23.914424 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.4.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.4.moving_variance W0922 19:54:23.914535 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.4.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.7.axis W0922 19:54:23.914642 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.7.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.7.gamma W0922 19:54:23.914747 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.7.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.7.beta W0922 19:54:23.914852 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.7.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.7.moving_mean W0922 19:54:23.914957 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.7.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.7.moving_variance W0922 19:54:23.915062 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.7.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.10.axis W0922 19:54:23.915168 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.10.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.10.gamma W0922 19:54:23.915274 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.10.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.10.beta W0922 19:54:23.915379 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.10.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.10.moving_mean W0922 19:54:23.915501 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.10.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.10.moving_variance W0922 19:54:23.915611 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.1.10.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.1.axis W0922 19:54:23.915718 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.1.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.1.gamma W0922 19:54:23.915823 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.1.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.1.beta W0922 19:54:23.915927 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.1.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.1.moving_mean W0922 19:54:23.916033 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.1.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.1.moving_variance W0922 19:54:23.916138 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.1.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.4.axis W0922 19:54:23.916244 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.4.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.4.gamma W0922 19:54:23.916350 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.4.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.4.beta W0922 19:54:23.916456 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.4.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.4.moving_mean W0922 19:54:23.916561 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.4.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.4.moving_variance W0922 19:54:23.916666 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.4.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.7.axis W0922 19:54:23.916772 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.7.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.7.gamma W0922 19:54:23.944508 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.7.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.7.beta W0922 19:54:23.944702 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.7.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.7.moving_mean W0922 19:54:23.944851 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.7.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.7.moving_variance W0922 19:54:23.944986 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.7.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.10.axis W0922 19:54:23.945124 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.10.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.10.gamma W0922 19:54:23.945355 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.10.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.10.beta W0922 19:54:23.945599 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.10.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.10.moving_mean W0922 19:54:23.945844 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.10.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.10.moving_variance W0922 19:54:23.946079 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.2.10.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.1.axis W0922 19:54:23.946323 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.1.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.1.gamma W0922 19:54:23.946470 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.1.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.1.beta W0922 19:54:23.946600 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.1.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.1.moving_mean W0922 19:54:23.946733 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.1.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.1.moving_variance W0922 19:54:23.946861 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.1.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.4.axis W0922 19:54:23.947021 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.4.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.4.gamma W0922 19:54:23.947158 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.4.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.4.beta W0922 19:54:23.947301 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.4.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.4.moving_mean W0922 19:54:23.947423 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.4.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.4.moving_variance W0922 19:54:23.947589 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.4.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.7.axis W0922 19:54:23.947717 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.7.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.7.gamma W0922 19:54:23.947844 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.7.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.7.beta W0922 19:54:23.947968 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.7.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.7.moving_mean W0922 19:54:23.948092 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.7.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.7.moving_variance W0922 19:54:23.948218 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.7.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.10.axis W0922 19:54:23.948343 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.10.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.10.gamma W0922 19:54:23.948468 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.10.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.10.beta W0922 19:54:23.948593 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.10.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.10.moving_mean W0922 19:54:23.948717 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.10.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.10.moving_variance W0922 19:54:23.948843 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.3.10.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.1.axis W0922 19:54:23.948968 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.1.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.1.gamma W0922 19:54:23.949093 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.1.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.1.beta W0922 19:54:23.949217 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.1.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.1.moving_mean W0922 19:54:23.949339 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.1.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.1.moving_variance W0922 19:54:23.949463 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.1.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.4.axis W0922 19:54:23.949588 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.4.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.4.gamma W0922 19:54:23.949712 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.4.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.4.beta W0922 19:54:23.949837 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.4.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.4.moving_mean W0922 19:54:23.949962 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.4.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.4.moving_variance W0922 19:54:23.950087 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.4.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.7.axis W0922 19:54:23.950213 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.7.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.7.gamma W0922 19:54:23.950339 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.7.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.7.beta W0922 19:54:23.950463 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.7.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.7.moving_mean W0922 19:54:23.950600 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.7.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.7.moving_variance W0922 19:54:23.950810 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.7.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.10.axis W0922 19:54:23.950945 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.10.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.10.gamma W0922 19:54:23.951074 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.10.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.10.beta W0922 19:54:23.951200 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.10.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.10.moving_mean W0922 19:54:23.951326 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.10.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.10.moving_variance W0922 19:54:23.951452 140036650096448 util.py:149] Unresolved object in checkpoint: (root).model._box_predictor._base_tower_layers_for_heads.class_predictions_with_background.4.10.moving_variance WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details. W0922 19:54:23.951614 140036650096448 util.py:157] A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details. WARNING:tensorflow:FailedPreconditionError: /home/dominik/Downloads/repo/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/checkpoint; Is a directory W0922 19:54:23.955974 140036650096448 checkpoint_management.py:295] FailedPreconditionError: /home/dominik/Downloads/repo/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/checkpoint; Is a directory WARNING:tensorflow:/home/dominik/Downloads/repo/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/checkpoint: Checkpoint ignored W0922 19:54:23.956107 140036650096448 checkpoint_management.py:296] /home/dominik/Downloads/repo/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/checkpoint: Checkpoint ignored WARNING:tensorflow:FailedPreconditionError: /home/dominik/Downloads/repo/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/checkpoint; Is a directory W0922 19:54:23.956307 140036650096448 checkpoint_management.py:295] FailedPreconditionError: /home/dominik/Downloads/repo/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/checkpoint; Is a directory WARNING:tensorflow:/home/dominik/Downloads/repo/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/checkpoint: Checkpoint ignored W0922 19:54:23.956394 140036650096448 checkpoint_management.py:296] /home/dominik/Downloads/repo/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/checkpoint: Checkpoint ignored Traceback (most recent call last): File "model_main_tf2.py", line 113, in <module> tf.compat.v1.app.run() File "/home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/tensorflow/python/platform/app.py", line 40, in run _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef) File "/home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/absl/app.py", line 300, in run _run_main(main, args) File "/home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/absl/app.py", line 251, in _run_main sys.exit(main(argv)) File "model_main_tf2.py", line 104, in main model_lib_v2.train_loop( File "/home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/object_detection/model_lib_v2.py", line 630, in train_loop manager.save() File "/home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/tensorflow/python/training/checkpoint_management.py", line 819, in save self._record_state() File "/home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/tensorflow/python/training/checkpoint_management.py", line 722, in _record_state update_checkpoint_state_internal( File "/home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/tensorflow/python/training/checkpoint_management.py", line 247, in update_checkpoint_state_internal file_io.atomic_write_string_to_file(coord_checkpoint_filename, File "/home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/tensorflow/python/lib/io/file_io.py", line 570, in atomic_write_string_to_file rename(temp_pathname, filename, overwrite) File "/home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/tensorflow/python/lib/io/file_io.py", line 529, in rename rename_v2(oldname, newname, overwrite) File "/home/dominik/anaconda3/envs/tensorflow/lib/python3.8/site-packages/tensorflow/python/lib/io/file_io.py", line 545, in rename_v2 _pywrap_file_io.RenameFile( tensorflow.python.framework.errors_impl.FailedPreconditionError: /home/dominik/Downloads/repo/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/checkpoint.tmpb8d6f3e0407948dfa546755fd5e7eb78; Is a directory
@domindominik I have the same error, I'll let you know once I've resolved it.
I think if you rename the "checkpoint" folder to e.g. "checkpoint_start" and adapt this in the config file as well it should work
@Pooya-Mohammadi-K I look forward to hearing from you
@map-mymories
I created cats_model directory, after starting the training:
fine_tune_checkpoint: "/home/dominik/Downloads/repo/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/cats_model/"
I got the following message:
ValueError: Couldn't find 'checkpoint' file or checkpoints in given directory /home/dominik/Downloads/repo/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/cats_model/
fine_tune_checkpoint: "/home/dominik/Downloads/repo/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/cats_model/ckpt-0"
tensorflow.python.framework.errors_impl.NotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for /home/dominik/Downloads/repo/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/cats_model/ckpt-0
did you copy the
- ckpt-0.data-00000-of-00001
- ckpt-0.index in the "cats_model" folder?
@map-mymories
After copying the files, I get the following message:
raise ValueError("Couldn't find 'checkpoint' file or checkpoints in " ValueError: Couldn't find 'checkpoint' file or checkpoints in given directory /home/dominik/Downloads/repo/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/cats_model/
@Pooya-Mohammadi-K @tombstone @jch1 @pkulzc Do you have any ideas what I should do to train a chosen model? If I need to check something additional, please inform me.
I'm having the same error in Colab.
Did anyone found a solution for this error?
@androuino
- What is the size of your photos for learning and testing the model?
- Have you tried using another model?
- Write something more about the environment
-
The size of the photos is 800x1440.
-
Two different models I have already tried.
-
The environment was tested in Colab, the dataset was mounted from Google drive, the browser I am using is Chrome.
@androuino
I have the same error. Using the pipeline config that is in the model tar and getting error with checkpoint restore.
nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2019 NVIDIA Corporation Built on Sun_Jul_28_19:12:52_Pacific_Daylight_Time_2019 Cuda compilation tools, release 10.1, V10.1.243
and with TF / Python version
`Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 27 2018, 03:37:03) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information.
import tensorflow 2021-02-03 09:51:52.180503: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll print(tensorflow.version) 2.2.0`
Using below
python model_main_tf2.py --pipeline_config_path=workspace/pipeline.config --model_dir=/workspace --alsologtostderr
with model and config from
http://download.tensorflow.org/models/object_detection/tf2/20200711/efficientdet_d0_coco17_tpu-32.tar.gz
I did change the number of classes for the tuning, but not sure if that is creating all these missing weights.
Error log
`...
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.0.0.1.1 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.0.0.1.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.0.0.1.2 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.0.0.1.2 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.1.0.1.0 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.1.0.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.2.1.1.0 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.2.1.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.0.1.0 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.0.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.0.1.1 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.0.1.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.1.1.0 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.1.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.0.1.0 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.0.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.0.1.1 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.0.1.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.1.1.0 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.1.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.0.1.0 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.0.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.0.1.1 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.0.1.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.1.1.0 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.1.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.0.1.0 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.0.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.0.1.1 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.0.1.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.2.1.0 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.2.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.0.1.0 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.0.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.0.1.1 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.0.1.1 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.2.1.0 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.2.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.8.2.1.0 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.8.2.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.9.1.1.0 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.9.1.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.10.1.1.0 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.10.1.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.11.1.1.0 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.11.1.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.12.1.1.0 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.12.1.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.13.1.1.0 W0203 08:00:33.702571 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.13.1.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.14.2.1.0 W0203 08:00:33.711225 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.14.2.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.15.2.1.0 W0203 08:00:33.711225 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.15.2.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.16.2.1.0 W0203 08:00:33.711225 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.16.2.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.17.1.1.0 W0203 08:00:33.711225 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.17.1.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.18.1.1.0 W0203 08:00:33.712229 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.18.1.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.19.1.1.0 W0203 08:00:33.712648 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.19.1.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.20.1.1.0 W0203 08:00:33.712792 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.20.1.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.21.1.1.0 W0203 08:00:33.712792 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.21.1.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.22.2.1.0 W0203 08:00:33.712792 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.22.2.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.23.2.1.0 W0203 08:00:33.712792 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.23.2.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.24.2.1.0 W0203 08:00:33.713803 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.24.2.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.25.1.1.0 W0203 08:00:33.713803 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.25.1.1.0 WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.0.0.1.0.kernel W0203 08:00:33.713803 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.0.0.1.0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.0.0.1.0.bias W0203 08:00:33.713803 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.0.0.1.0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.0.0.1.1.axis W0203 08:00:33.714803 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.0.0.1.1.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.0.0.1.1.gamma W0203 08:00:33.714803 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.0.0.1.1.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.0.0.1.1.beta W0203 08:00:33.714803 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.0.0.1.1.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.0.0.1.1.moving_mean W0203 08:00:33.714803 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.0.0.1.1.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.0.0.1.1.moving_variance W0203 08:00:33.715803 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.0.0.1.1.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.0.1.0.kernel W0203 08:00:33.715803 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.0.1.0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.0.1.0.bias W0203 08:00:33.715803 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.0.1.0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.0.1.1.axis W0203 08:00:33.715803 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.0.1.1.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.0.1.1.gamma W0203 08:00:33.715803 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.0.1.1.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.0.1.1.beta W0203 08:00:33.716802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.0.1.1.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.0.1.1.moving_mean W0203 08:00:33.716802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.0.1.1.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.0.1.1.moving_variance W0203 08:00:33.716802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.3.0.1.1.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.0.1.0.kernel W0203 08:00:33.716802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.0.1.0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.0.1.0.bias W0203 08:00:33.717801 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.0.1.0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.0.1.1.axis W0203 08:00:33.717801 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.0.1.1.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.0.1.1.gamma W0203 08:00:33.717801 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.0.1.1.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.0.1.1.beta W0203 08:00:33.718802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.0.1.1.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.0.1.1.moving_mean W0203 08:00:33.718802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.0.1.1.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.0.1.1.moving_variance W0203 08:00:33.718802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.4.0.1.1.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.0.1.0.kernel W0203 08:00:33.718802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.0.1.0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.0.1.0.bias W0203 08:00:33.719802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.0.1.0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.0.1.1.axis W0203 08:00:33.719802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.0.1.1.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.0.1.1.gamma W0203 08:00:33.719802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.0.1.1.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.0.1.1.beta W0203 08:00:33.719802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.0.1.1.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.0.1.1.moving_mean W0203 08:00:33.720802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.0.1.1.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.0.1.1.moving_variance W0203 08:00:33.720802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.5.0.1.1.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.0.1.0.kernel W0203 08:00:33.720802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.0.1.0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.0.1.0.bias W0203 08:00:33.720802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.0.1.0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.0.1.1.axis W0203 08:00:33.721805 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.0.1.1.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.0.1.1.gamma W0203 08:00:33.721805 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.0.1.1.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.0.1.1.beta W0203 08:00:33.721805 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.0.1.1.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.0.1.1.moving_mean W0203 08:00:33.721805 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.0.1.1.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.0.1.1.moving_variance W0203 08:00:33.722801 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.6.0.1.1.moving_variance WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.0.1.0.kernel W0203 08:00:33.722801 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.0.1.0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.0.1.0.bias W0203 08:00:33.722801 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.0.1.0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.0.1.1.axis W0203 08:00:33.722801 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.0.1.1.axis WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.0.1.1.gamma W0203 08:00:33.722801 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.0.1.1.gamma WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.0.1.1.beta W0203 08:00:33.723802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.0.1.1.beta WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.0.1.1.moving_mean W0203 08:00:33.723802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.0.1.1.moving_mean WARNING:tensorflow:Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.0.1.1.moving_variance W0203 08:00:33.723802 12320 util.py:144] Unresolved object in checkpoint: (root).model._feature_extractor._bifpn_stage.node_input_blocks.7.0.1.1.moving_variance WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details. W0203 08:00:33.724805 12320 util.py:152] A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.`
Deleting files in model dir before training solved the issue. In my case, I changed SSD Model to EfficientDet D0 model and did not delete the model_dir files, which gave me the same error.
If you still have the "FailedPreconditionError: ../ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/checkpoint.tmpb8d6f3e0407948dfa546755fd5e7eb78; Is a directory" Error, Check whether you have called the model_main script with the right model_dir, it probably should be something like ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/saved_model and not only ssd_resnet50_v1_fpn_1024x1024_coco17_tpu-8/ at least that worked for me.
