蜷缩的蜗牛
蜷缩的蜗牛

Kyverno 快速体验

kbsonlong
 约 1700 字 预计阅读 4 分钟  

安装 CLI 

1
2
3

export version=1.6.0
wget https://github.com/kyverno/kyverno/releases/download/v${version}/kyverno-cli_v${version}_darwin_arm64.tar.gz
tar zxf kyverno-cli_v${version}_darwin_arm64.tar.gz

通过标签匹配命名空间 Pods

此处包含两个规则，两个规则联动

  1. 根据命名空间标签匹配 Pod, 然后注入注解 inject-lxcfs=enabled
  2. 通过 preconditions 前置条件匹配注解 inject-lxcfs=enabled 的 Pod 注入 lxcfs 卷，开启 Lxcfs；

从而达到以下效果

  • 针对命名空间下所有部署开启 lxcfs: kubectl label ns default lxcfs-admission-webhook=enabled
  • 针对具体某个部署开启 lxcfs: kubectl annotate –overwrite pods <pod_name> inject-lxcfs=enabled
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167

cat << EOF | kubectl apply -f - 
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  annotations:
    policies.kyverno.io/category: Sample
    policies.kyverno.io/description: >-
      Match the Pod injection annotation inject-lxcfs=enabled in the label inject-lxcfs=enabled namespace;
      # kubectl label ns default lxcfs-admission-webhook=enabled
      Match the Pod annotation inject-lxcfs=enabled
      are injected into the lxcfs storage volume;
      # kubectl annotate --overwrite pods <pod_name> inject-lxcfs=enabled
    policies.kyverno.io/minversion: 1.6.0
    policies.kyverno.io/subject: Pod, Volume
    policies.kyverno.io/title: Add Lxcfs to Pod
  name: enabled-lxcfs
spec:
  background: true
  rules:
  - match:
      any:
      - resources:
          kinds:
          - Pod
          namespaceSelector:
            matchExpressions:
            - key: lxcfs-admission-webhook
              operator: In
              values:
              - enabled
    mutate:
      patchStrategicMerge:
        metadata:
          annotations:
            inject-lxcfs: enabled
    name: namespace-pod-add-lxcfs
  - match:
      any:
      - resources:
          kinds:
          - Pod
    mutate:
      patchesJson6902: |-
        - op: add
          path: /spec/volumes/-
          value:
            hostPath:
              path: /var/lib/lxc/lxcfs/proc/cpuinfo
              type: ""
            name: lxcfs-proc-cpuinfo
        - op: add
          path: /spec/containers/0/volumeMounts/-
          value:
            mountPath: /proc/cpuinfo
            mountPropagation: HostToContainer
            name: lxcfs-proc-cpuinfo

        - op: add
          path: /spec/volumes/-
          value:
            hostPath:
              path: /var/lib/lxc/lxcfs/proc/diskstats
              type: ""
            name: lxcfs-proc-diskstats
        - op: add
          path: /spec/containers/0/volumeMounts/-
          value:
            mountPath: /proc/diskstats
            mountPropagation: HostToContainer
            name: lxcfs-proc-diskstats

        - op: add
          path: /spec/volumes/-
          value:
            hostPath:
              path: /var/lib/lxc/lxcfs/proc/meminfo
              type: ""
            name: lxcfs-proc-meminfo
        - op: add
          path: /spec/containers/0/volumeMounts/-
          value:
            mountPath: /proc/meminfo
            mountPropagation: HostToContainer
            name: lxcfs-proc-meminfo

        - op: add
          path: /spec/volumes/-
          value:
            hostPath:
              path: /var/lib/lxc/lxcfs/proc/stat
              type: ""
            name: lxcfs-proc-stat
        - op: add
          path: /spec/containers/0/volumeMounts/-
          value:
            mountPath: /proc/stat
            mountPropagation: HostToContainer
            name: lxcfs-proc-stat

        - op: add
          path: /spec/volumes/-
          value:
            hostPath:
              path: /var/lib/lxc/lxcfs/proc/swaps
              type: ""
            name: lxcfs-proc-swaps
        - op: add
          path: /spec/containers/0/volumeMounts/-
          value:
            mountPath: /proc/swaps
            mountPropagation: HostToContainer
            name: lxcfs-proc-swaps

        - op: add
          path: /spec/volumes/-
          value:
            hostPath:
              path: /var/lib/lxc/lxcfs/proc/uptime
              type: ""
            name: lxcfs-proc-uptime
        - op: add
          path: /spec/containers/0/volumeMounts/-
          value:
            mountPath: /proc/uptime
            mountPropagation: HostToContainer
            name: lxcfs-proc-uptime

        - op: add
          path: /spec/volumes/-
          value:
            hostPath:
              path: /var/lib/lxc/lxcfs/proc/loadavg
              type: ""
            name: lxcfs-proc-loadavg
        - op: add
          path: /spec/containers/0/volumeMounts/-
          value:
            mountPath: /proc/loadavg
            mountPropagation: HostToContainer
            name: lxcfs-proc-loadavg

        - op: add
          path: /spec/volumes/-
          value:
            hostPath:
              path: /var/lib/lxc/lxcfs/sys/devices/system/cpu/online
              type: ""
            name: lxcfs-sys-devices-system-cpu-online
        - op: add
          path: /spec/containers/0/volumeMounts/-
          value:
            mountPath: /sys/devices/system/cpu/online
            mountPropagation: HostToContainer
            name: lxcfs-sys-devices-system-cpu-online
    name: add-volume
    preconditions:
      all:
      - key: '{{request.object.metadata.annotations."inject-lxcfs" || ""}}'
        operator: Equals
        value: enabled
      - key: "{{request.operation || 'BACKGROUND'}}"
        operator: AnyIn
        value:
        - CREATE
        - UPDATE
  validationFailureAction: Audit
EOF

根据注解调整调度器名称 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

cat << EOF | kubectl apply -f -
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: replace-scheduler
  annotations:
    policies.kyverno.io/title: Replace Scheduler Name
    policies.kyverno.io/category: Sample
    policies.kyverno.io/subject: Pod, Scheduler
    policies.kyverno.io/minversion: 1.6.0
spec:
  rules:
  - name: replace-scheduler
    match:
      any:
      - resources:
          annotations:
            aquaman.shopline.com/schedulername: "?*"
          kinds:
          - Pod
    preconditions:
      any:
      - key: "{{request.operation || 'BACKGROUND'}}"
        operator: AnyIn
        value:
        - CREATE
        - UPDATE
    mutate:
      patchesJson6902: |-
        - op: replace
          path: /spec/schedulerName
          value: '{{request.object.metadata.annotations."aquaman.shopline.com/schedulername"}}'
EOF
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

cat << EOF | kubectl apply -f - 
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: disable-service-links
  annotations:
    policies.kyverno.io/title: Disable  Service  Links Variables
    policies.kyverno.io/category: Mutate
    policies.kyverno.io/subject: Pod
    policies.kyverno.io/minversion: 1.6.0
    policies.kyverno.io/description: >-
      A Pod enableServiceLinks entry defines fields ,Add enableServiceLinks and disable Service makeLinkVariables.
spec:
  rules:
  - name: disable-service-links
    match:
      any:
      - resources:
          kinds:
          - Pod
    preconditions:
      any:
      - key: "{{request.operation || 'BACKGROUND'}}"
        operator: AnyIn
        value:
        - CREATE
        - UPDATE
    mutate:
     patchesJson6902: |-
        - op: replace
          path: /spec/enableServiceLinks
          value: false
EOF

替换镜像仓库 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: prepend-registry
  annotations:
    pod-policies.kyverno.io/autogen-controllers: none
    policies.kyverno.io/title: Replace Image Registry
    policies.kyverno.io/category: Sample
    policies.kyverno.io/severity: medium
    policies.kyverno.io/subject: Pod
    policies.kyverno.io/minversion: 1.5.4
    policies.kyverno.io/description: >-
      Rather than blocking Pods which come from outside registries,
      it is also possible to mutate them so the pulls are directed to
      approved registries. In some cases, those registries may function as
      pull-through proxies and can fetch the image if not cached.
      This policy policy mutates all images either
      in the form 'image:tag' or 'registry.corp.com/image:tag' to be
      myregistry.corp.com/. Any path in the image name will be preserved.
spec:
  background: false
  rules:
    - name: replace-image-registry-pod-containers
      match:
        resources:
          kinds:
          - Pod
      preconditions:
        all:
        - key: '{{request.operation || ''BACKGROUND''}}'
          operator: AnyIn
          value:
          - CREATE
      mutate:
        foreach:
        - list: "request.object.spec.containers"
          patchStrategicMerge:
            spec:
              imagePullSecrets:
              - name: zj-repo-sg-aws
              - name: ee-sl-repo-sg
              - name: ee-sl-repo-us
              containers:
              - name: "{{ element.name }}"
                image: |-
                        {{ regex_replace_all('.*.ap-southeast-1.cr.aliyuncs.com/', '{{element.image}}', 'kbsonlong.us-east-1.cr.aliyuncs.com/$1') }}
    - name: replace-image-registry-pod-initcontainers
      match:
        resources:
          kinds:
          - Pod
      preconditions:
        all:
        - key: "{{ request.object.spec.initContainers[] || '' | length(@) }}"
          operator: GreaterThanOrEquals
          value: 1
        - key: '{{request.operation || ''BACKGROUND''}}'
          operator: AnyIn
          value:
          - CREATE
      mutate:
        foreach:
        - list: "request.object.spec.initContainers"
          patchStrategicMerge:
            spec:
              imagePullSecrets:
              - name: zj-repo-sg-aws
              - name: ee-sl-repo-sg
              - name: ee-sl-repo-us
              initContainers:
              - name: "{{ element.name }}"
                image: |-
                        {{ regex_replace_all('.*.ap-southeast-1.cr.aliyuncs.com/', '{{element.image}}', 'kbsonlong.us-east-1.cr.aliyuncs.com/$1') }}
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: replace-image-registry
  annotations:
    pod-policies.kyverno.io/autogen-controllers: none
    policies.kyverno.io/title: Replace Image Registry
    policies.kyverno.io/category: Sample
    policies.kyverno.io/severity: medium
    policies.kyverno.io/subject: Pod
    policies.kyverno.io/minversion: 1.5.4
    policies.kyverno.io/description: >-
      除了阻止来自外部注册表的 Pod，以便将拉取定向到已批准的注册表。
      在某些情况下，这些注册表可能充当拉取代理，并且可以在未缓存的情况下获取图像。
      此策略将所有镜像变异为'image:tag' 或 'registry.kbsonlong.com/image:tag' 形式，
      为`myregistry.kbsonlong.com/`。镜像名称中的任何路径都将被保留。     
spec:
  background: false
  rules:
    - name: replace-image-registry-pod-containers
      match:
        resources:
          kinds:
          - Pod
      mutate:
        foreach:
        - list: "request.object.spec.containers"
          patchStrategicMerge:
            spec:
              containers:
              - name: "{{ element.name }}"
                image: |-    
                        {{ regex_replace_all('^[^/]+', '{{element.image}}', 'repo-us-registry.us-east-1.cr.aliyuncs.com' )}}
    - name: replace-image-registry-pod-initcontainers
      match:
        resources:
          kinds:
          - Pod
      preconditions:
        all:
        - key: "{{ request.object.spec.initContainers[] || '' | length(@) }}"
          operator: GreaterThanOrEquals
          value: 1
      mutate:
        foreach:
        - list: "request.object.spec.initContainers"
          patchStrategicMerge:
            spec:
              initContainers:
              - name: "{{ element.name }}"
                image: |-    
                        {{ regex_replace_all('^[^/]+', '{{element.image}}', 'repo-us-registry.us-east-1.cr.aliyuncs.com' )}}
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

cat << EOF | kubectl apply -f -
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: prepend-registry-init
spec:
  background: false
  rules:
  - name: prepend-registry-containers
    match:
      any:
      - resources:
          kinds:
          - Pod
    preconditions:
      any:
      - key: '{{request.operation || ''BACKGROUND''}}'
        operator: AnyIn
        value:
        - CREATE
    mutate:
      foreach:
      - list: "request.object.spec.initContainers"
        patchStrategicMerge:
          spec:
            initContainers:
            - name: "{{ element.name }}"           
              image: repo-us-registry.us-east-1.cr.aliyuncs.com/seam/{{ images.initContainers."{{element.name}}".name}}:{{images.initContainers."{{element.name}}".tag}}
---
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: prepend-registry
spec:
  background: false
  rules:
  - name: prepend-registry-containers
    match:
      any:
      - resources:
          kinds:
          - Pod
    preconditions:
      any:
      - key: '{{request.operation || ''BACKGROUND''}}'
        operator: AnyIn
        value:
        - CREATE
    mutate:
      foreach:
      - list: "request.object.spec.containers"
        patchStrategicMerge:
          spec:
            containers:
            - name: "{{ element.name }}"           
              image: repo-us-registry.us-east-1.cr.aliyuncs.com/seam/{{ images.containers."{{element.name}}".name}}:{{images.containers."{{element.name}}".tag}}
EOF

禁止创建

reference

  1. https://kyverno.io/policies/other/add-certificates-volume/add-certificates-volume/

  2. https://aws.amazon.com/cn/blogs/china/implementing-policy-as-code-through-kyverno-on-amazon-eks/

  3. https://kyverno.io/docs/writing-policies/match-exclude/

  4. https://release-1-6-0.kyverno.io/docs/kyverno-cli/#apply

更新于 2024-04-09 
CC BY-NC 4.0
阅读原始文档
LINE Myspace
返回 | 主页
0%