Kubernetes 污点和容忍度

Posted by Vito on January 25, 2024

污点 taint

  • 污点效果 effect
    • NoSchedule
      • 除非具有匹配的容忍度规约,否则新的 Pod 一定不会被调度到带有该污点的节点上。 当前正在节点上运行的 Pod 不会被驱逐。
    • PreferNoSchedule
      • 除非具有匹配的容忍度规约,否则新的 Pod 尽量不会被调度到带有该污点的节点上。
    • NoExecute
      • 如果节点上已存在的 Pod 不能容忍这类污点,会马上被驱逐。
      • 如果节点上已存在的 Pod 能够容忍这类污点,且在容忍度定义中没有指定 tolerationSeconds, 则 Pod 还会一直在这个节点上运行。
      • 如果节点上已存在的 Pod 能够容忍这类污点,而且指定了 tolerationSeconds, 则 Pod 还能在这个节点上继续运行这个指定的时间长度。 这段时间过去后,节点生命周期控制器从节点驱除这些 Pod。
  • 给 Node 添加污点:kubectl taint nodes node1 key1=value1:NoSchedule
1
2
3
4
5
6
7
8
[root@k8s-master1 ~]# kubectl taint no k8s-node1.zhch.lan memory=pressure:NoSchedule
node/k8s-node1.zhch.lan tainted

[root@k8s-master1 ~]# kubectl taint no k8s-node1.zhch.lan network=unavailable:NoExecute
node/k8s-node1.zhch.lan tainted

[root@k8s-master1 ~]# kubectl taint no k8s-node1.zhch.lan unreachable:NoSchedule
node/k8s-node1.zhch.lan tainted
  • 查询已存在的污点
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
[root@k8s-master1 ~]# kubectl describe nodes k8s-node1.zhch.lan 
Name:               k8s-node1.zhch.lan
Roles:              <none>
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=k8s-node1.zhch.lan
                    kubernetes.io/os=linux
Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/cri-dockerd.sock
                    node.alpha.kubernetes.io/ttl: 0
                    projectcalico.org/IPv4Address: 192.168.99.204/24
                    projectcalico.org/IPv4IPIPTunnelAddr: 10.244.37.192
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Wed, 13 Dec 2023 00:26:45 +0800
Taints:             network=unavailable:NoExecute
                    memory=pressure:NoSchedule
                    unreachable:NoSchedule
Unschedulable:      false
1
2
3
4
5
[root@k8s-master1 ~]# kubectl describe nodes k8s-node2.zhch.lan 
Taints:             <none>

[root@k8s-master1 ~]# kubectl describe nodes k8s-master1.zhch.lan 
Taints:             node-role.kubernetes.io/control-plane:NoSchedule
  • 删除污点
1
2
3
4
5
6
7
8
[root@k8s-master1 ~]# kubectl taint no k8s-node1.zhch.lan memory=pressure:NoSchedule-
node/k8s-node1.zhch.lan untainted

[root@k8s-master1 ~]# kubectl taint no k8s-node1.zhch.lan network-
node/k8s-node1.zhch.lan untainted

[root@k8s-master1 ~]# kubectl taint no k8s-node1.zhch.lan unreachable-
node/k8s-node1.zhch.lan untainted

容忍度 tolerations

  • tolerations.operator
    • Equal(默认值)
      • key value effect 必须与 Taint 的设置保持一致
    • Exists
      • 此时容忍度不能指定 value,只需要 key 和 effect 与 Taint 的设置保持一致
    • 特殊情形1:空的 key 配合 Exists 操作符能够匹配所有的键和值
    • 特殊情形2:空的 effect 匹配所有的 effect
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
[root@k8s-master1 test]# vim deploy-taint-test.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deploy-taint-test
spec:
  replicas: 5
  selector:
    matchLabels:
      app: deploy-vito-nginx
  template: 
    metadata:
      labels:
        app: deploy-vito-nginx
    spec:
      tolerations:
      - key: "memory"
        operator: "Equal"
        value: "pressure"
        effect: "NoSchedule"
      - key: "network"
        operator: "Exists"
        effect: "NoExecute"
        tolerationSeconds: 600
      - key: "unreachable"
        operator: "Exists"
      containers:
      - image: nginx:1.7.9
        imagePullPolicy: IfNotPresent
        name: deploy-taint-test-c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[root@k8s-master1 ~]# kubectl describe nodes k8s-node1.zhch.lan
Taints:             network=unavailable:NoExecute
                    memory=pressure:NoSchedule
                    unreachable:NoSchedule

[root@k8s-master1 test]# kubectl create -f deploy-taint-test.yaml 
deployment.apps/deploy-taint-test created

[root@k8s-master1 test]# kubectl get po -o wide
NAME                                READY   STATUS    RESTARTS   AGE   IP              NODE                 NOMINATED NODE   READINESS GATES
deploy-taint-test-84b68b8df-4m5fn   1/1     Running   0          31s   10.244.8.248    k8s-node2.zhch.lan   <none>           <none>
deploy-taint-test-84b68b8df-bc5bc   1/1     Running   0          31s   10.244.37.207   k8s-node1.zhch.lan   <none>           <none>
deploy-taint-test-84b68b8df-sxbv2   1/1     Running   0          31s   10.244.8.252    k8s-node2.zhch.lan   <none>           <none>
deploy-taint-test-84b68b8df-vsfw8   1/1     Running   0          31s   10.244.37.209   k8s-node1.zhch.lan   <none>           <none>
deploy-taint-test-84b68b8df-z44hx   1/1     Running   0          31s   10.244.37.208   k8s-node1.zhch.lan   <none>           <none>