[BUG] rshift '>>' operator doesn't work properly with remoteEntities

[andresgomezfrr]

Describe the bug

The rshift >> operator doesn't work properly using remoteEntities. Even if you define remote_entity_1 >> remote_entity_2 the upstream is not configured and both entities run in parallel.

More context:

The stacktrace to create a remoteEntitiy is the following one: https://github.com/flyteorg/flytekit/blob/master/flytekit/remote/remote_callable.py#L35 --> https://github.com/flyteorg/flytekit/blob/master/flytekit/remote/remote_callable.py#L53 --> https://github.com/flyteorg/flytekit/blob/master/flytekit/remote/remote_callable.py#L33 --> https://github.com/flyteorg/flytekit/blob/master/flytekit/core/promise.py#L930 --> https://github.com/flyteorg/flytekit/blob/master/flytekit/core/promise.py#L1027 --> https://github.com/flyteorg/flytekit/blob/master/flytekit/core/promise.py#L659-L676

In the last link you get the promise of the remoteEntity and build a Output helper, but you can check how the >> is not implemented in this Output helper: https://github.com/flyteorg/flytekit/blob/master/flytekit/core/promise.py#L665-L676

        def runs_before(self, other: Any):
            """
            This function is just here to allow local workflow execution to run. See the corresponding function in
            flytekit.core.node.Node for more information. Local workflow execution in the manual ``create_node``
            paradigm is already determined by the order in which the nodes were created.
            """
            # TODO: If possible, add a check and raise an Exception if create_node was not called in the correct order.
            return self

        def __rshift__(self, other: Any):
            # See comment for runs_before
            return other

You can workaround this issue doing something like:

remote_entity_1.output1.ref.node >> remote_entity_2.output1.ref.node

But I think it is not expected for the users.

Expected behavior

remote_entity_1 >> remote_entity_2

This should define the upstreams properly to remote_entity_2

Additional context to reproduce

No response

Screenshots

No response

Are you sure this issue hasn't been raised already?

• [X] Yes

Have you read the Code of Conduct?

• [X] Yes

[andresgomezfrr]

cc @pingsutw

[runllm[bot]]

The issue you are describing is related to the improper functioning of the >> operator with remote entities in Flyte. This problem has been noted in various contexts, including chaining tasks and workflows. According to this discussion, chaining a task with a remote task that doesn't have outputs doesn't work correctly. Additionally, another discussion highlights issues with the >> operator in different scenarios. Furthermore, a related GitHub issue discusses the order of execution not being respected when running locally.

Would you like more detail on how to work around this issue or any other specific aspect?

Sources:

• Hi everyone I want to report a possible issue that I ve dete
• I have found a peculiar case where I can not successfully re
• GitHub Issue #3644

---

This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @runllm to ask follow-up questions. See runllm.com for more!

[Mecoli1219]

@andresgomezfrr Could you provide an example like the one you have just mentioned?

remote_entity_1.output1.ref.node >> remote_entity_2.output1.ref.node

[andresgomezfrr]

I tested this again with the latest release and looks like it is working fine, I'm going to close this issue