matrixone icon indicating copy to clipboard operation
matrixone copied to clipboard

fix shuffle join for recursive cte

Open iamlinjunhong opened this issue 7 months ago • 2 comments

User description

What type of PR is this?

  • [ ] API-change
  • [ ] BUG
  • [ ] Improvement
  • [ ] Documentation
  • [ ] Feature
  • [ ] Test and CI
  • [ ] Code Refactoring

Which issue(s) this PR fixes:

issue #https://github.com/matrixorigin/MO-Cloud/issues/5764

What this PR does / why we need it:

fix shuffle join for recursive cte


PR Type

Bug fix


Description

  • Fix shuffle join logic for recursive CTE queries

  • Add proper batch handling for last batch condition

  • Include comprehensive test case for recursive CTE with large dataset


Changes walkthrough 📝

Relevant files
Bug fix
shuffle.go
Fix batch handling in shuffle operations                                 

pkg/sql/colexec/shuffle/shuffle.go

  • Fix batch handling logic by adding condition for bat.Last()
  • Resolve merge conflict by removing duplicate import
  • Ensure proper flow control for last batch in shuffle operations
  • +4/-0     
    Tests
    recursive_cte.result
    Update test results for recursive CTE                                       

    test/distributed/cases/recursive_cte/recursive_cte.result

  • Add test results for new recursive CTE test case
  • Include expected output for large dataset query with 6 rows
  • Update test results to match new test scenarios
  • +8/-1     
    recursive_cte.sql
    Add large dataset recursive CTE test                                         

    test/distributed/cases/recursive_cte/recursive_cte.sql

  • Add comprehensive test case with large dataset (1M rows)
  • Include recursive CTE query with JOIN operations
  • Test tenant filtering in recursive queries
  • +6/-1     

    Need help?
  • Type /help how to ... in the comments thread for any questions about Qodo Merge usage.
  • Check out the documentation for more information.
  • iamlinjunhong avatar Jun 25 '25 10:06 iamlinjunhong

    You are nearing your monthly Qodo Merge usage quota. For more information, please visit here.

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
    🧪 PR contains tests
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Merge Conflict

    The code contains unresolved merge conflict markers that need to be cleaned up before merging. The import section has Git conflict markers that will cause compilation errors.

    <<<<<<< HEAD
    	"fmt"
    
    =======
    >>>>>>> 74ada5848 (fix shuffle join for recursive cte)
    
    Logic Error

    The control flow logic appears incorrect. When bat.Last() is true, the function returns immediately without setting proper status or handling the last batch condition, potentially causing incomplete processing.

    } else if bat.Last() {
    	return result, nil
    

    qodo-code-review[bot] avatar Jun 25 '25 10:06 qodo-code-review[bot]

    You are nearing your monthly Qodo Merge usage quota. For more information, please visit here.

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Impact
    Possible issue
    Remove merge conflict markers

    Remove the merge conflict markers that were accidentally committed. These
    markers should be resolved during the merge process and not left in the final
    code.

    pkg/sql/colexec/shuffle/shuffle.go [19-23]

    -<<<<<<< HEAD
     	"fmt"
     
    -=======
    ->>>>>>> 74ada5848 (fix shuffle join for recursive cte)
    -
    
    • [ ] Apply / Chat <!-- /improve --apply_suggestion=0 -->
    Suggestion importance[1-10]: 10

    __

    Why: The suggestion correctly identifies leftover merge conflict markers (<<<<<<< HEAD, =======, >>>>>>>) in the code. This is a critical error that will cause a compilation failure and must be fixed.

    High
    • [ ] Update <!-- /improve_multi --more_suggestions=true -->

    qodo-code-review[bot] avatar Jun 25 '25 10:06 qodo-code-review[bot]