Kubernetes cluster have a default scheduler kube-scheduler. If the default scheduler does not suits our requirement we can also create our own scheduler. In the post we will discus how to create multiple scheduler and schedule pods based on different scheduler.

Setup

I am using the Virtualbox(running in Ubuntu 18.04 physical machine) for this entire setup . The physical machine is Dell inspiron laptop with 12GB RAM , Intel® Core™ i7-6500U CPU @ 2.50GHz × 4 and 512GB SSD hardisk.

Step 1: Copy the default scheduler yaml and modify

Copy the default scheduler yaml from master node and modify the name.

In this below example i am naming the custom schedule are my-scheduler. Add a ServiceAccount and ClusterRoleBinding to the yaml file as given below.

The key changes made are

  • line 27: name: my-scheduler
  • line 30: serviceAccountName: my-scheduler
  • line 38: - --leader-elect=false
  • line 39: - --scheduler-name=my-scheduler
vikki@kubernetes1:~$ sudo cp /etc/kubernetes/manifests/kube-scheduler.yaml custom-scheduler.yaml
vikki@kubernetes1:~$ sudo chmod 777 custom-scheduler.yaml 
vikki@kubernetes1:~$ vim custom-scheduler.yaml 
Step 2: Create the custom scheduler
vikki@kubernetes1:~$ kubectl create -f custom-scheduler.yaml 
serviceaccount/my-scheduler created
clusterrolebinding.rbac.authorization.k8s.io/my-scheduler-as-kube-scheduler created
pod/my-scheduler created
vikki@kubernetes1:~$ kubectl get pods -n=kube-system 
NAME                                      READY   STATUS    RESTARTS   AGE
calico-kube-controllers-55754f75c-fgs8g   1/1     Running   7          22d
calico-node-4h72l                         1/1     Running   7          22d
calico-node-ld84s                         1/1     Running   7          22d
calico-node-lrfz9                         1/1     Running   2          31h
calico-node-ws576                         1/1     Running   1          27h
coredns-5644d7b6d9-2g6rs                  1/1     Running   7          22d
coredns-5644d7b6d9-ccxsg                  1/1     Running   7          22d
etcd-kubernetes1                          1/1     Running   8          22d
kube-apiserver-kubernetes1                1/1     Running   8          22d
kube-controller-manager-kubernetes1       1/1     Running   8          22d
kube-proxy-6xd8l                          1/1     Running   2          31h
kube-proxy-96q5x                          1/1     Running   7          22d
kube-proxy-njl6r                          1/1     Running   7          22d
kube-proxy-whlhw                          1/1     Running   1          27h
kube-scheduler-kubernetes1                1/1     Running   8          22d
my-scheduler                              1/1     Running   0          6s
Now we can see a new custom scheduler my-scheduler is running along with defautl scheduler kube-scheduler-kubernetes1
Step 3: Edit the custerrole for kube-scheduler

If RBAC is enabled on your cluster, you must update the system:kube-scheduler cluster role. Edit the clusterrole for kube-scheduler and modify.

Below are the key changes made in orignal file

  • line 33: - my-scheduler
  • line 131: - storageclasses
vikki@kubernetes1:~$ kubectl edit clusterrole system:kube-scheduler
Step 4: Create pods with different types of scheduler

Create a pod without explicitly mentioning any scheuler name. This pod should be scheduled by default scheduler kube-scheduler

vikki@kubernetes1:~$ vim pod_default_scheduler.yaml

Create a pod by explicitly mentioning custom scheuler name my-scheduler.

vikki@kubernetes1:~$ vim pod_custom_scheduler.yaml 
vikki@kubernetes1:~$ kubectl create -f pod_default_scheduler.yaml 
pod/nginx-pod-default-scheduler created
vikki@kubernetes1:~$ kubectl create -f pod_custom_scheduler.yaml 
pod/nginx-pod-custom-scheduler created
vikki@kubernetes1:~$ kubectl get pods nginx-pod-custom-scheduler nginx-pod-default-scheduler
NAME                          READY   STATUS    RESTARTS   AGE
nginx-pod-custom-scheduler    1/1     Running   0          9m9s
nginx-pod-default-scheduler   1/1     Running   0          9m13s
vikki@kubernetes1:~$ 
Now we can see both the pods are created and running
vikki@kubernetes1:~$ kubectl get events -o wide |grep nginx-pod-custom-scheduler |head -1
<unknown>   Normal   Scheduled          pod/nginx-pod-custom-scheduler                                                                    my-scheduler            Successfully assigned default/nginx-pod-custom-scheduler to kubernetes3            <unknown>    0       nginx-pod-custom-scheduler.15da76383c2c2859

vikki@kubernetes1:~$ kubectl get events -o wide |grep nginx-pod-default-scheduler |head -1
<unknown>   Normal   Scheduled          pod/nginx-pod-default-scheduler                                                                   default-scheduler       Successfully assigned default/nginx-pod-default-scheduler to kubernetes3           <unknown>    0       nginx-pod-default-scheduler.15da76372f4444f7
From the events, we can see pod nginx-pod-custom-scheduler is scheduled by my-scheduler and nginx-pod-default-scheduler by  default-scheduler.