models icon indicating copy to clipboard operation
models copied to clipboard

No OpKernel was registered to support Op 'TPUOrdinalSelector' used by node TPUOrdinalSelector

Open Jconn opened this issue 6 years ago • 9 comments

System information

  • What is the top-level directory of the model you are using:
  • Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
  • TensorFlow installed from (source or binary): have tried both, first binary install and then from source
  • TensorFlow version (use command below): 1.14
  • Bazel version (if compiling from source): 0.26.1
  • CUDA/cuDNN version:10.0.130
  • GPU model and memory: NVIDIA GeForce GTX 1080 8GB Ram
  • Exact command to reproduce: python3 object_detection/tpu_exporters/export_saved_model_tpu.py

I have trained a model on my GPU and want to export the model to a TPU for inference.

I am running the script found at

models/research/object_detection/tpu_exporters/export_saved_model_tpu.py

When I run the below command: python3 object_detection/tpu_exporters/export_saved_model_tpu.py --pipeline_config_file=object_detection/output/models/model/ssd_resnet50_v1_fpn_shared_box_predictor_640x640_coco14_sync.config --ckpt_path=object_detection/output/models/model/model.ckpt-39688 --export_dir=/tmp/out --input_type="image_tensor" --input_placeholder_name=image_tensor:0 I get the following exception:

With the full exception being 
Traceback (most recent call last):
  File "/home/johnconn/.local/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call
    return fn(*args)
  File "/home/johnconn/.local/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1348, in _run_fn
    self._extend_graph()
  File "/home/johnconn/.local/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1388, in _extend_graph
    tf_session.ExtendSession(self._session)
tensorflow.python.framework.errors_impl.InvalidArgumentError: No OpKernel was registered to support Op 'TPUOrdinalSelector' used by {{node TPUOrdinalSelector}}with these attrs: []
Registered devices: [CPU, GPU]
Registered kernels:
  <no registered kernels>

	 [[TPUOrdinalSelector]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "object_detection/tpu_exporters/export_saved_model_tpu.py", line 54, in <module>
    tf.app.run()
  File "/home/johnconn/.local/lib/python3.6/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/johnconn/.local/lib/python3.6/site-packages/absl/app.py", line 300, in run
    _run_main(main, args)
  File "/home/johnconn/.local/lib/python3.6/site-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "object_detection/tpu_exporters/export_saved_model_tpu.py", line 47, in main
    FLAGS.input_type, FLAGS.use_bfloat16)
  File "/home/johnconn/utils/models/research/object_detection/tpu_exporters/export_saved_model_tpu_lib.py", line 80, in export
    sess.run(init_op)
  File "/home/johnconn/.local/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 956, in run
    run_metadata_ptr)
  File "/home/johnconn/.local/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1180, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/johnconn/.local/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1359, in _do_run
    run_metadata)
  File "/home/johnconn/.local/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1384, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: No OpKernel was registered to support Op 'TPUOrdinalSelector' used by node TPUOrdinalSelector (defined at /home/johnconn/.local/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py:1748) with these attrs: []
Registered devices: [CPU, GPU]
Registered kernels:
  <no registered kernels>

	 [[TPUOrdinalSelector]]

It looks like @shreyaaggarwal encountered the same issue, posting it here https://github.com/tensorflow/models/issues/4283

Jconn avatar Aug 26 '19 18:08 Jconn