Problem to export model from PyTorch to TensorFlow

[Nicolas-Gsln]

Hi, Thank you for your work, I find it really useful and I am trying to embed it for a test in a real-time environment. In order to do that, I want to export the model to TensorFlowLite, so that I could do small changes (like quantization) which is more efficient than doing them with PyTorch. To export the model to TFlite, I first exported it to ONNX and now I'm trying to export it from ONNX to TF with onnx-tf library. I am using opset_version=11, the lowest version compatible with all the PyTorch operations in CFNet.

However I faced many problems in my journey, first I had a dimension problem with the conversion to ONNX so I decided to use a fixed input size for the images (wh = 512768). I tested the results with the Middlebury SDK (I have done a resize on the input images, run my ONNX model and then resized the disparity maps I get) and these results are quite good.

Then to export to TF, I first had an issue with an unsupported operation :

RuntimeError: Resize coordinate_transformation_mode=pytorch_half_pixel is not supported in Tensorflow.

I tried to add "align_corners=True" inside the upsample functions in the model code, and it solved the problem

Right now I am facing an other issue but I didn't find any way to solve it, here are the logs :

Traceback (most recent call last):
  File "/path/scripts/../export_TF.py", line 17, in <module>
 
   tf_rep.export_graph("%s/%s.pb" % (args.outdir,input_model_name))

  File "/path/onnx-tensorflow/onnx_tf/backend_rep.py", line 143, in export_graph

signatures=self.tf_module.__call__.get_concrete_function(

 File "/path/venv/lib/python3.9/site-packages/tensorflow/python/eager/def_function.py", line 1264, in get_concrete_function

    concrete = self._get_concrete_function_garbage_collected(*args, **kwargs)

 File "/path/venv/lib/python3.9/site-packages/tensorflow/python/eager/def_function.py", line 1244, in _get_concrete_function_garbage_collected

    self._initialize(args, kwargs, add_initializers_to=initializers)

  File "/path/venv/lib/python3.9/site-packages/tensorflow/python/eager/def_function.py", line 785, in _initialize

    self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access

  File "/path/venv/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 2983, in _get_concrete_function_internal_garbage_collected

    graph_function, _ = self._maybe_define_function(args, kwargs)

  File "/path/venv/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 3292, in _maybe_define_function

    graph_function = self._create_graph_function(args, kwargs)

  File "/path/venv/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 3130, in _create_graph_function func_graph_module.func_graph_from_py_func(

  File "/path/venv/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py", line 1161, in func_graph_from_py_func

    func_outputs = python_func(*func_args, **func_kwargs)

  File "/path/venv/lib/python3.9/site-packages/tensorflow/python/eager/def_function.py", line 677, in wrapped_fn

    out = weak_wrapped_fn().__wrapped__(*args, **kwds)

  File "/path/venv/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 3831, in bound_method_wrapper

    return wrapped_fn(*args, **kwargs)

  File "/path/venv/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py", line 1147, in autograph_handler

    raise e.ag_error_metadata.to_exception(e)

ValueError: in user code:

    File "/path/onnx-tensorflow/onnx_tf/backend_tf_module.py", line 99, in __call__  *

        output_ops = self.backend._onnx_node_to_tensorflow_op(onnx_node,

    File "/path/onnx-tensorflow/onnx_tf/backend.py", line 347, in _onnx_node_to_tensorflow_op  *

        return handler.handle(node, tensor_dict=tensor_dict, strict=strict)

    File "/path/onnx-tensorflow/onnx_tf/handlers/handler.py", line 58, in handle  *

        cls.args_check(node, **kwargs)

    File "/path/onnx-tensorflow/onnx_tf/handlers/backend/resize.py", line 68, in args_check  *

        x_shape = x.get_shape().as_list()

ValueError: as_list() is not defined on an unknown TensorShape.

Using netron.app, I found that the node 10454 seemed to have a dimension problem (which corresponds to the upsample operation at the line 660 of cfnet.py), so I tried to hardcode all the dimensions with my input size :

pred1_s2 = F.upsample(pred1_s2 * 2, [512, 768], mode='bilinear', align_corners=True)

but it didn't resolve my problem at all, and I really don't have any idea on how to solve it. My TF version is 2.8.0

Did you already tried (and succeeded) to export the model to TensorFlow, and if so how did you do it ? If not, do you have any idea on how I could solve this problem ?

Thank you.