Add feature to use the node selector in the scheduler to filter BM Nodes with GPUs

[haitwang-cloud]

1. Issue or feature description

Description

In our cluster, we have both VM nodes and BM (Bare Metal) nodes. However, only the BM nodes have GPUs. Therefore, I would like to request a new feature that reads the current scheduler's node selector and uses this node selector to filter BM nodes.

Rationale

This feature will help optimize our scheduling by ensuring that tasks requiring GPUs are only scheduled on BM nodes that actually have GPUs, thereby improving resource allocation and performance.

Proposed Implementation

• Modify the scheduler to read the node selector configuration.
• Implement a filter mechanism that uses the node selector to identify BM nodes.
• Ensure that only BM nodes with GPUs are considered for tasks requiring GPU resources.

Benefits

• Improved scheduling efficiency.
• Better resource utilization.
• Enhanced performance for GPU-dependent tasks.