flytekit icon indicating copy to clipboard operation
flytekit copied to clipboard

fix: add FlyteValueException handling and clean up exit_handler calls

Open vincent0426 opened this issue 1 year ago • 8 comments

Tracking issue

Why are the changes needed?

Currently, if a task returns a value but does not specify a return type, the return value will be empty instead of raising an error.

It is expected to raise an error if a function returns a value but no return type is specified.

This should throw an error, but it doesn't

pyflyte run example.py greeting_length --greeting="hello"

@task()
def greeting_length(greeting: str):
    """A task the counts the length of a greeting."""
    return len(greeting)
Screenshot 2025-01-05 at 8 57 25 PM

What changes were proposed in this pull request?

Add conditions check to throw an error if native_outputs have value but no output interface is specified

How was this patch tested?

Setup process

pyflyte run example.py greeting_length --greeting="hello"

@task()
def greeting_length(greeting: str):
    """A task the counts the length of a greeting."""
    return len(greeting)

Screenshots

Screenshot 2025-01-05 at 8 58 41 PM

Check all the applicable boxes

  • [ ] I updated the documentation accordingly.
  • [X] All new and existing tests passed.
  • [X] All commits are signed-off.

Related PRs

Docs link

Summary by Bito

Implementation of enhanced error handling for tasks with undefined/unspecified return types, introducing FlyteValueException for validation of native outputs. Added comprehensive test cases covering regular returns, optional types, and error scenarios. Enhanced test coverage for task return type handling mechanisms and improved validation in test files to prevent silent failures. Optimized exit_handler calls in VSCode and Jupyter environments for better type safety.

Unit tests added: True

Estimated effort to review (1-5, lower is better): 1

vincent0426 avatar Jan 06 '25 05:01 vincent0426

Code Review Agent Run #a5368c

Actionable Suggestions - 1
  • flytekit/interactive/vscode_lib/decorator.py - 1
    • Consider handling exit_handler return value · Line 459-459
Review Details
  • Files reviewed - 3 · Commit Range: ea17a4d..ea17a4d
    • flytekit/core/base_task.py
    • flytekit/interactive/vscode_lib/decorator.py
    • plugins/flytekit-flyteinteractive/flytekitplugins/flyteinteractive/jupyter_lib/decorator.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by Bito Logo

flyte-bot avatar Jan 06 '25 05:01 flyte-bot

Changelist by Bito

This pull request implements the following key changes.

Key Change Files Impacted
Bug Fix - Enhanced Error Handling for Task Return Types

base_task.py - Added FlyteValueException handling for tasks with undefined return types

test_task_return.py - Added new test cases for task return type validation

Feature Improvement - Type Safety Enhancements

test_elastic_task.py - Added return type annotation for test_task

test_plugin.py - Added complete return type annotation for DataFrame function

generic_idl_flytetypes.py - Added return type annotation for DC type

msgpack_idl_flytetypes.py - Added return type annotation for DC type

Other Improvements - Exit Handler Optimization

test_flyteinteractive_jupyter.py - Added return_value parameter to exit_handler mock

test_flyteinteractive_vscode.py - Added return_value parameter to exit_handler mock

test_flyteinteractive_vscode.py - Added return_value parameter to exit_handler mock

flyte-bot avatar Jan 06 '25 05:01 flyte-bot

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 79.54%. Comparing base (0ad84f3) to head (912cf34). Report is 10 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3036      +/-   ##
==========================================
- Coverage   82.79%   79.54%   -3.26%     
==========================================
  Files           3      202     +199     
  Lines         186    21359   +21173     
  Branches        0     2746    +2746     
==========================================
+ Hits          154    16989   +16835     
- Misses         32     3602    +3570     
- Partials        0      768     +768     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Jan 08 '25 01:01 codecov[bot]

Code Review Agent Run #0101b5

Actionable Suggestions - 0
Additional Suggestions - 1
  • plugins/flytekit-flyteinteractive/tests/test_flyteinteractive_jupyter.py - 1
    • Consider necessity of explicit None return · Line 31-31
Review Details
  • Files reviewed - 4 · Commit Range: ea17a4d..60f4732
    • flytekit/interactive/vscode_lib/decorator.py
    • plugins/flytekit-flyteinteractive/flytekitplugins/flyteinteractive/jupyter_lib/decorator.py
    • plugins/flytekit-flyteinteractive/tests/test_flyteinteractive_jupyter.py
    • tests/flytekit/unit/interactive/test_flyteinteractive_vscode.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by Bito Logo

flyte-bot avatar Jan 08 '25 01:01 flyte-bot

Code Review Agent Run #f8b810

Actionable Suggestions - 0
Review Details
  • Files reviewed - 3 · Commit Range: 60f4732..f1d19c8
    • plugins/flytekit-flyteinteractive/tests/test_flyteinteractive_vscode.py
    • plugins/flytekit-kf-pytorch/tests/test_elastic_task.py
    • plugins/flytekit-pandera/tests/test_plugin.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by Bito Logo

flyte-bot avatar Jan 08 '25 03:01 flyte-bot

Code Review Agent Run #d37b87

Actionable Suggestions - 0
Review Details
  • Files reviewed - 2 · Commit Range: f1d19c8..c6d70d7
    • tests/flytekit/integration/remote/workflows/basic/generic_idl_flytetypes.py
    • tests/flytekit/integration/remote/workflows/basic/msgpack_idl_flytetypes.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by Bito Logo

flyte-bot avatar Jan 08 '25 05:01 flyte-bot

@vincent0426 Can you add some unit tests to make sure running the task/workflow without specifying output while returning a non-None value would throw an error? Also, running with --remote flag should also throw an error, but I think you can fix it in another PR.

Mecoli1219 avatar Jan 10 '25 18:01 Mecoli1219

Code Review Agent Run #dd9258

Actionable Suggestions - 0
Review Details
  • Files reviewed - 1 · Commit Range: c6d70d7..912cf34
    • tests/flytekit/unit/core/test_task_return.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by Bito Logo

flyte-bot avatar Jan 11 '25 02:01 flyte-bot