matrixone-operator icon indicating copy to clipboard operation
matrixone-operator copied to clipboard

[Feature Request]: How to schedule mo services in a proper way in order to aovid resource fragmentation

Open aronchanisme opened this issue 2 years ago • 0 comments

Is there an existing issue for the same feature request?

  • [X] I have checked the existing issues.

Is your feature request related to a problem?

Currently component of mo services are started in an order like this by mo operator: logservice -> dn -> proxy -> cn(or cn groups)

We have a situation here. Let's say, we have a cluster 5 nodes, among which 3 are labeled for logservice, 1 is labeled for dn, and the rest is labeld for cn, but no nodes are labled for proxy. host1-3: log host4: dn host5: cn

Let's assume the 3 nodes labled for logservice are of big size, while the one labled for cn is of a relatively small size.

Imagine if proxy has no nodeSelector and k8s assigned it to node 5, then cn might not be able to be assigned to this node because node 5 might not have enough resources for cn anymore.

Describe the feature you'd like

In a cluster which is designed to have just enough resources for an mo cluster requires, mo operator should make sure all components of mo services are assigned properly to the right nodes.

Describe implementation you've considered

Not sure if changing the start order will help solve this issue. Maybe we should consider how to scheduler the big 'guy' first, then scheduler the smaller ones after that?

Documentation, Adoption, Use Case, Migration Strategy

n.a.

Additional information

n.a.

aronchanisme avatar Jun 09 '23 07:06 aronchanisme