02 - Multiple Schedulers

Kubernetes Multiple Schedulers

📌 為什麼需要多個 Scheduler？

Kubernetes 預設使用 kube-scheduler 處理所有 Pod 的排程任務。

但某些場景下，我們可能需要：

• 使用不同策略排程不同類型的工作負載（ex: ML Job、批次任務）
• 為特定隊列、優先級、資源需求定制排程邏輯
• 測試自定義排程演算法（研究用）

⚙️ 如何實作 Multiple Schedulers？

在 cluster 中「同時部署多個 scheduler」，然後讓特定 Pod 使用你指定的 scheduler 名稱進行排程。

1️⃣ 設計你的排程架構

假設你要：

• 使用原生 default-scheduler 處理一般 workload
• 使用 custom-scheduler 處理 ML/批次任務
• 未來可能擴充更多 scheduler

你可以這樣規劃：

2️⃣ 撰寫或部署自定義 Scheduler

可以是：

1. Fork 官方 kube-scheduler 修改排程邏輯（進階）
2. 用 Go/Python 等語言實作一個簡單的排程邏輯
3. 或使用現成的開源方案（如 Volcano、YuniKorn）

建立自定義 Scheduler 後，打包為 Docker Image。

3️⃣ 部署到 Kubernetes

以一個最簡單的 custom scheduler 為例（custom-scheduler-deploy.yaml）：

```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: custom-scheduler
spec:
  replicas: 1
  selector:
    matchLabels:
      component: custom-scheduler
  template:
    metadata:
      labels:
        component: custom-scheduler
    spec:

      serviceAccountName: default
      containers:
      - name: custom-scheduler
        image: yourname/custom-scheduler:latest
        env:

        - name: IN_CLUSTER

          value: "true"

```

$ kubectl apply -f custom-scheduler-deploy.yaml               

4️⃣ 建立測試 `example-pod` Pod 使用自訂 Scheduler

```
apiVersion: v1
kind: Pod
metadata:
  name: example-pod
spec:
  schedulerName: custom-scheduler
  containers:
    - name: busybox
      image: busybox
      command: ["sleep", "3600"]

```

5️⃣ 驗證 Pod 是否被正確排程

$ kubectl get pods -o wide               

你應該會看到 Pod 被指派到某個節點，而不是卡在 Pending。

We have already created the `ServiceAccount` and `ClusterRoleBinding` that our custom scheduler will make use of.

Checkout the following Kubernetes objects:

• ServiceAccount: my-scheduler (kube-system namespace)
• ClusterRoleBinding: my-scheduler-as-kube-scheduler
• ClusterRoleBinding: my-scheduler-as-volume-scheduler

$ kubectl get sa my-scheduler -n kube-system             

$ kubectl get serviceaccount -n kube-system               
$ kubectl get sa -n kube-system                                 

ServiceAccount: my-scheduler

$ kubectl get clusterrolebinding                     

ClusterRoleBinding: my-scheduler-as-kube-scheduler

ClusterRoleBinding: my-scheduler-as-volume-scheduler

Q: Let's create a configmap that the new scheduler will employ using the concept of `ConfigMap as a volume`.

We have already given a configMap definition file called `my-scheduler-configmap.yaml` at `/root/` path that will create a configmap with name `my-scheduler-config` using the content of file `/root/my-scheduler-config.yaml`.

Use the imperative command `kubectl create -f` to create the configMap from the given definition file.

$ kubectl create -f /root/my-scheduler-configmap.yaml            

or

$ kubectl create configmap my-scheduler-config --from-file=/root/my-scheduler-config.yaml -n kube-system           

$ kubectl get configmap my-scheduler-config -n kube-system           

/root/my-scheduler-configmap.yaml

$ kubectl get configmap -A                    

Q: Deploy an additional scheduler to the cluster following the given specification.

Use the manifest file provided at `/root/my-scheduler.yaml`. 

Use the same image as used by the default kubernetes scheduler.

• Name: my-scheduler
• Status: Running
• Correct image used?

Use the file at `/root/my-scheduler.yaml` to create your own scheduler with correct image.

$ kubectl describe pod kube-scheduler-controlplane --namespace=kube-system | grep -i 'image'                

Image:         registry.k8s.io/kube-scheduler:v1.32.0

$ vim /root/my-scheduler.yaml                        

$ kubectl create -f my-scheduler.yaml              

/root/my-scheduler.yaml

Q: A POD definition file is given. Use it to create a POD with the new custom scheduler.

• Uses custom scheduler
• Status: Running

File is located at `/root/nginx-pod.yaml`.

Set `schedulerName` property on pod specification to the name of the new scheduler.

$ vim /root/nginx-pod.yaml                    

$ kubectl create -f nginx-pod.yaml          

/root/nginx-pod.yaml