matrixone icon indicating copy to clipboard operation
matrixone copied to clipboard
verified publisher icon matrixorigin

[Subtask]: refactor blocking class operators to avoid OOM

Open ouyuanning opened this issue 2 years ago • 3 comments

Parent Issue

#10729

筛一遍所有的阻塞类算子，避免mpool申请过多内存的问题和内存占用过多问题 目前自己预计要处理的算子包含： 1、mergeorder 2、lock_op 3、on_duplicate_key

其他join相关的算子，由龙冉通过spill那个issue处理。这里处理非join相关的阻塞类算子

Detail of Subtask

This issue is a summary of related task issues:

  • [x] #12956
  • [ ] #12021
  • [ ] https://github.com/matrixorigin/MO-Cloud/issues/1921
  • [ ] https://github.com/matrixorigin/MO-Cloud/issues/496
  • [ ] #14038

Describe implementation you've considered

No response

Additional information

No response

ouyuanning avatar Dec 15 '23 11:12 ouyuanning

可以先筛选出要改造的算子清单。 然后挨个看使用什么方案处理，看是否有可以抽象的

ouyuanning avatar Dec 20 '23 06:12 ouyuanning

待进一步讨论和spill issue的分工和方案 https://github.com/matrixorigin/matrixone/issues/13640

florashi181 avatar Dec 28 '23 07:12 florashi181

与内存issue 关联 https://github.com/matrixorigin/matrixone/issues/11509

florashi181 avatar Dec 28 '23 07:12 florashi181

在spill的issue跟踪，这里先关闭

ouyuanning avatar Oct 31 '24 02:10 ouyuanning