[Docs] Docstring for `map_task`'s `min_success_ratio`

[artgillespie]

Description

See Slack thread for more context.

I found the docstring

:param min_success_ratio: If specified, this determines the minimum fraction of total jobs which must complete
            successfully before terminating this task and marking it unsuccessful

to be somewhat unclear. Specifically, map_task will map across all inputs as long as min_success_ratio is satisfied and only terminate early when the number of failed tasks exceeds 1.0 - min_success_ratio

Are you sure this issue hasn't been raised already?

• [X] Yes

Have you read the Code of Conduct?

• [X] Yes

[welcome[bot]]

Thank you for opening your first issue here! 🛠

[dosubot[bot]]

🤖

Hello @artgillespie! I'm Dosu, a friendly bot here to help you while we wait for a real human maintainer. I'm here to assist with bugs, answer questions, and guide you through contributing. Remember, no question is too small or too big for me! Let's work together on this.

Thank you for your feedback on the map_task function's min_success_ratio parameter. I understand that the current docstring might be unclear, and I appreciate your suggestion for improvement.

Based on the context provided, the min_success_ratio parameter is used to determine the minimum ratio of successful sub-tasks required for the overall task to be considered successful. This is particularly useful in scenarios where a task is being executed in parallel across multiple nodes, and it's acceptable for a certain percentage of those nodes to fail without causing the entire task to fail.

In the Flyte codebase, the min_success_ratio parameter is used in the TestHandleArrayNodePhaseExecuting function. Here, it is used to determine whether the array node phase should transition to succeeding or failing based on the success ratio of the subnodes. For example, in the test case named "OneSubNodeSuccedeedMinSuccessRatio", the min_success_ratio is set to 0.5. This means that at least 50% of the subnodes must succeed for the array node to be considered successful. In this test case, one subnode succeeds and one fails, so the success ratio is 0.5, which meets the min_success_ratio. Therefore, the expected array node phase is v1alpha1.ArrayNodePhaseSucceeding.

In contrast, in the test case named "OneSubNodeFailed", there is no min_success_ratio set, so the default behavior is that all subnodes must succeed for the array node to be considered successful. In this test case, one subnode fails, so the expected array node phase is v1alpha1.ArrayNodePhaseFailing.

You can find these test cases in the flytepropeller/pkg/controller/nodes/array/handler_test.go file.

Based on this, a more accurate docstring for the min_success_ratio parameter could be:

:param min_success_ratio: This determines the minimum ratio of successful sub-tasks required for the overall task to be considered successful. If the success ratio of the sub-tasks falls below this value, the task will be terminated and marked as unsuccessful.

I hope this clarifies the purpose and usage of the min_success_ratio parameter. If you have any further questions or suggestions, please feel free to share them.

Sources

• flytepropeller/pkg/controller/nodes/array/handler_test.go

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[0xSolanaceae]

Is this something I can work on?

[davidmirror-ops]

@0xSolanaceae sure! let us know soon if you have questions. Thanks!

[github-actions[bot]]

"Hello 👋, this issue has been inactive for over 90 days. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will close the issue if we detect no activity in the next 7 days. Thank you for your contribution and understanding! 🙏"

[github-actions[bot]]

Hello 👋, this issue has been inactive for over 90 days and hasn't received any updates since it was marked as stale. We'll be closing this issue for now, but if you believe this issue is still relevant, please feel free to reopen it. Thank you for your contribution and understanding! 🙏