ALT Orchestra

Тема

Первичная установка ​

Для каждого хоста ALT Orchestra получить название свободного диска, куда будет установлен дистрибутив:

console
$ for i in 2 3 4 5 6; \
    do talosctl -e 192.168.1.$i -n 192.168.1.$i get disks --insecure; done

Примерный вывод:

text
NODE   NAMESPACE   TYPE   ID      VERSION   SIZE     READ ONLY   TRANSPORT   ROTATIONAL   WWID   MODEL           SERIAL
       runtime     Disk   loop0   2         4.1 kB   true
       runtime     Disk   loop1   2         98 MB    true
       runtime     Disk   sda     2         16 GB    false       virtio      true                QEMU HARDDISK
       runtime     Disk   sr0     2         377 MB   false       sata                            QEMU DVD-ROM

Получить версии образов:

console
REPO=p11 && \
KUBELET_VERSION="v1.32.7" && \
COREDNS_VERSION="$(curl -ks "https://registry.altlinux.org/v2/${REPO}/coredns/tags/list" | jq -r  '.tags | sort | .[]' | tail -1)" && \
ETCD_VERSION="$(curl -ks "https://registry.altlinux.org/v2/${REPO}/etcd/tags/list" | jq -r  '.tags | sort | .[]' | tail -1)" && \
PAUSE_VERSION="$(curl -ks "https://registry.altlinux.org/v2/${REPO}/pause/tags/list" | jq -r  '.tags | sort | .[]' | tail -1)"

Задать основные переменные:

console
DEVICE="/dev/sda" && \
INSTALLERIMAGE="altlinux.space/alt-orchestra/installer:<VERSION>"

Получить названия образов для основных контейнеров:

console
KUBELETIMAGE="registry.altlinux.org/${REPO}/kubelet:${KUBELET_VERSION}" && \
APISERVERIMAGE="registry.altlinux.org/${REPO}/kube-apiserver:${KUBELET_VERSION}" && \
CONTROLMANAGERIMAGE="registry.altlinux.org/${REPO}/kube-controller-manager:${KUBELET_VERSION}" && \
SHEDULERIMAGE="registry.altlinux.org/${REPO}/kube-scheduler:${KUBELET_VERSION}" && \
COREDNSIMAGE="registry.altlinux.org/${REPO}/coredns:${COREDNS_VERSION}" && \
ETCDIMAGE="registry.altlinux.org/${REPO}/etcd:${ETCD_VERSION}" && \
PAUSEIMAGE="registry.altlinux.org/${REPO}/pause:${PAUSE_VERSION}"

Запулить их:

console
$ for image in ${KUBELETIMAGE} ${APISERVERIMAGE} ${CONTROLMANAGERIMAGE} ${SHEDULERIMAGE} ${COREDNSIMAGE} ${ETCDIMAGE} ${PAUSEIMAGE} ${INSTALLERIMAGE}; \
    do podman pull --tls-verify=false $image; done

Проверить их наличие в кэше:

console
$ curl http://localhost:5000/v2/_catalog | jq 
$ curl http://localhost:5001/v2/_catalog | jq

Создать патч для worker в контексте ALT Orchestra:

console
$ cat > worker.patch << PATCH
.machine.features.kubePrism.enabled |= true |
.machine.features.kubePrism.port |= 7445 |
.cluster.discovery.enabled |= true |
.cluster.discovery.registries.kubernetes.disabled |= true |
.cluster.discovery.registries.service.endpoint |= "http://192.168.1.1:3000" |
.machine.install.disk="$DEVICE" |
.machine.kubelet.image|="$KUBELETIMAGE" |
.machine.install.image|="$INSTALLERIMAGE" |
.machine.registries.config."registry.altlinux.org".tls.insecureSkipVerify|=true |
.machine.registries.mirrors."registry.altlinux.org".endpoints |= ["http://192.168.1.1:5000"] |
.machine.registries.config."altlinux.space".tls.insecureSkipVerify|=true |
.machine.registries.mirrors."altlinux.space".endpoints |= ["http://192.168.1.1:5001"] |
.machine.registries.config."ghcr.io".tls.insecureSkipVerify|=true |
.machine.registries.mirrors."ghcr.io".endpoints |= ["http://192.168.1.1:5002"] |
.machine.registries.config."registry.k8s.io".tls.insecureSkipVerify|=true |
.machine.registries.mirrors."registry.k8s.io".endpoints |= ["http://192.168.1.1:5003"] |
.machine.registries.config."docker.io".tls.insecureSkipVerify|=true |
.machine.registries.mirrors."docker.io".endpoints |= ["http://192.168.1.1:5004"] |
.machine.registries.config."factory.altlinux.space".tls.insecureSkipVerify|=true |
.machine.registries.mirrors."factory.altlinux.space".endpoints |= ["http://192.168.1.1:5005"] |
.machine.files |= [{"content": "[pause]
[plugins.\"io.containerd.cri.v1.images\".pinned_images]
sandbox = \"$PAUSEIMAGE\"","path": "/etc/cri/conf.d/20-customization.part","op": "create"}]
PATCH

Создать патч для controplane в контексте ALT Orchestra:

console
$ cat > controlplane.patch << PATCH
.machine.features.kubePrism.enabled |= true |
.machine.features.kubePrism.port |= 7445 |
.machine.install.disk="$DEVICE" |
.machine.kubelet.image|="$KUBELETIMAGE" |
.machine.install.image|="$INSTALLERIMAGE" |
.machine.registries.config."registry.altlinux.org".tls.insecureSkipVerify|=true |
.machine.registries.mirrors."registry.altlinux.org".endpoints |= ["http://192.168.1.1:5000"] |
.machine.registries.config."altlinux.space".tls.insecureSkipVerify|=true |
.machine.registries.mirrors."altlinux.space".endpoints |= ["http://192.168.1.1:5001"] |
.machine.registries.config."ghcr.io".tls.insecureSkipVerify|=true |
.machine.registries.mirrors."ghcr.io".endpoints |= ["http://192.168.1.1:5002"] |
.machine.registries.config."registry.k8s.io".tls.insecureSkipVerify|=true |
.machine.registries.mirrors."registry.k8s.io".endpoints |= ["http://192.168.1.1:5003"] |
.machine.registries.config."docker.io".tls.insecureSkipVerify|=true |
.machine.registries.mirrors."docker.io".endpoints |= ["http://192.168.1.1:5004"] |
.machine.registries.config."factory.altlinux.space".tls.insecureSkipVerify|=true |
.machine.registries.mirrors."factory.altlinux.space".endpoints |= ["http://192.168.1.1:5005"] |
.machine.files |= [{"content": "[pause]
[plugins.\"io.containerd.cri.v1.images\".pinned_images]
sandbox = \"$PAUSEIMAGE\"","path": "/etc/cri/conf.d/20-customization.part","op": "create"}] |
.cluster.discovery.enabled |= true |
.cluster.discovery.registries.kubernetes.disabled |= true |
.cluster.discovery.registries.service.endpoint |= "http://192.168.1.1:3000" |
.cluster.apiServer.image|="$APISERVERIMAGE" |
.cluster.controllerManager.image|="$CONTROLMANAGERIMAGE" |
.cluster.scheduler.image|="$SHEDULERIMAGE" |
.cluster.etcd.image|="$ETCDIMAGE" |
.cluster.coreDNS.image|="$COREDNSIMAGE"
PATCH

Сгенерировать конфигурацию ALT Orchestra по умолчанию (без Kube Proxy):

console
$ talosctl gen config talos https://192.168.1.2:6443 --force --config-patch '{"cluster":{"network":{"cni":{"name":"none"}},"proxy":{"disabled":true}}}'

Скопировать конфигурацию ALT Orchestra в пути по умолчанию:

console
$ mkdir -p ~/.talos
$ mv ~/talosconfig ~/.talos/config

Создать конфигурацию для ALT Orchestra:

console
$ cat controlplane.yaml | yq -y "$(cat controlplane.patch)" > orchestra_controlplane.yaml
$ cat worker.yaml | yq -y "$(cat worker.patch)" > orchestra_worker.yaml

Применить конфигурацию для каждой машин Controlplane:

console
$ talosctl apply-config --insecure -n 192.168.1.2 --file orchestra_controlplane.yaml
$ talosctl apply-config --insecure -n 192.168.1.3 --file orchestra_controlplane.yaml
$ talosctl apply-config --insecure -n 192.168.1.4 --file orchestra_controlplane.yaml

Применить конфигурацию для каждой машин Worker:

console
$ talosctl apply-config --insecure -n 192.168.1.5 --file orchestra_worker.yaml
$ talosctl apply-config --insecure -n 192.168.1.6 --file orchestra_worker.yaml

Выполнить bootstrap (создание кластера etcd):

console
$ talosctl -e 192.168.1.2 -n 192.168.1.2 bootstrap

Создать конфигурацию Kubernetes из Talosctl:

console
$ talosctl -e 192.168.1.2 -n 192.168.1.2 kubeconfig

Установить Cilium через Helm:

console
$ curl https://altlinux.space/cloud/charts/raw/branch/master/cilium/sisyphus/1.18.2/values.yaml -o cilium-values.yaml
$ sed -i '/genericDigest: "sha256:a573bf42c0199aef9c68b657b2ea53cc31293a0a6eb2e812604cc8c31f846db0"/d' cilium-values.yaml
$ sed -i "s|useDigest: true|useDigest: false|g" cilium-values.yaml
$ helm repo add cilium https://helm.cilium.io
$ helm repo update
$ helm install \
    cilium \
    cilium/cilium \
    --version 1.18.2 \
    --namespace kube-system \
    --set ipam.mode=kubernetes \
    --set kubeProxyReplacement=true \
    --set securityContext.capabilities.ciliumAgent="{CHOWN,KILL,NET_ADMIN,NET_RAW,IPC_LOCK,SYS_ADMIN,SYS_RESOURCE,DAC_OVERRIDE,FOWNER,SETGID,SETUID}" \
    --set securityContext.capabilities.cleanCiliumState="{NET_ADMIN,SYS_ADMIN,SYS_RESOURCE}" \
    --set cgroup.autoMount.enabled=false \
    --set cgroup.hostRoot=/sys/fs/cgroup \
    --set k8sServiceHost=localhost \
    --set k8sServicePort=7445 \
    -f cilium-values.yaml

Убедиться, что узлы готовы:

console
$ kubectl get nodes

Примерный вывод:

text
NAME                    STATUS   ROLES           AGE   VERSION
alt-orchestra-2n9-fm5   Ready    <none>          69s   v1.32.7
alt-orchestra-4ty-xyj   Ready    control-plane   70s   v1.32.7
alt-orchestra-861-o83   Ready    <none>          61s   v1.32.7
alt-orchestra-9v7-0p2   Ready    control-plane   63s   v1.32.7
alt-orchestra-kus-sds   Ready    control-plane   67s   v1.32.7

Получить список всех подов в кластере:

console
$ kubectl get pods -A

Примерный вывод:

text
NAMESPACE     NAME                                            READY   STATUS    RESTARTS       AGE
kube-system   cilium-2jvb4                                    1/1     Running   0              112s
kube-system   cilium-4xrlg                                    1/1     Running   1 (52s ago)    112s
kube-system   cilium-9nspx                                    1/1     Running   0              112s
kube-system   cilium-9zb6v                                    1/1     Running   0              113s
kube-system   cilium-cp96k                                    1/1     Running   0              113s
kube-system   cilium-envoy-2m8wp                              1/1     Running   0              112s
kube-system   cilium-envoy-4mx5h                              1/1     Running   0              113s
kube-system   cilium-envoy-g8qh5                              1/1     Running   0              112s
kube-system   cilium-envoy-xffdn                              1/1     Running   0              113s
kube-system   cilium-envoy-z8xw6                              1/1     Running   0              113s
kube-system   cilium-operator-67f4b4f5fd-9djtg                1/1     Running   1 (82s ago)    112s
kube-system   cilium-operator-67f4b4f5fd-wkjdq                1/1     Running   1 (69s ago)    113s
kube-system   coredns-5966c6bdcd-2gxvq                        1/1     Running   0              3m43s
kube-system   coredns-5966c6bdcd-qfmn2                        1/1     Running   0              3m43s
kube-system   kube-apiserver-alt-orchestra-27m-03d            1/1     Running   0              3m30s
kube-system   kube-apiserver-alt-orchestra-gpg-qkl            1/1     Running   0              3m36s
kube-system   kube-apiserver-alt-orchestra-urk-gud            1/1     Running   0              2m59s
kube-system   kube-controller-manager-alt-orchestra-27m-03d   1/1     Running   0              3m30s
kube-system   kube-controller-manager-alt-orchestra-gpg-qkl   1/1     Running   1 (96s ago)    3m36s
kube-system   kube-controller-manager-alt-orchestra-urk-gud   1/1     Running   2 (4m3s ago)   2m59s
kube-system   kube-scheduler-alt-orchestra-27m-03d            1/1     Running   0              3m30s
kube-system   kube-scheduler-alt-orchestra-gpg-qkl            1/1     Running   0              3m36s
kube-system   kube-scheduler-alt-orchestra-urk-gud            1/1     Running   4 (58s ago)    2m58s

Вывести список всех ресурсов во всех пространствах имён кластера Kubernetes:

console
$ kubectl get all -A

Примерный вывод:

text
NAMESPACE     NAME                                                READY   STATUS    RESTARTS        AGE
kube-system   pod/cilium-2jvb4                                    1/1     Running   0               2m11s
kube-system   pod/cilium-4xrlg                                    1/1     Running   1 (71s ago)     2m11s
kube-system   pod/cilium-9nspx                                    1/1     Running   0               2m11s
kube-system   pod/cilium-9zb6v                                    1/1     Running   0               2m12s
kube-system   pod/cilium-cp96k                                    1/1     Running   0               2m12s
kube-system   pod/cilium-envoy-2m8wp                              1/1     Running   0               2m11s
kube-system   pod/cilium-envoy-4mx5h                              1/1     Running   0               2m12s
kube-system   pod/cilium-envoy-g8qh5                              1/1     Running   0               2m11s
kube-system   pod/cilium-envoy-xffdn                              1/1     Running   0               2m12s
kube-system   pod/cilium-envoy-z8xw6                              1/1     Running   0               2m12s
kube-system   pod/cilium-operator-67f4b4f5fd-9djtg                1/1     Running   1 (101s ago)    2m11s
kube-system   pod/cilium-operator-67f4b4f5fd-wkjdq                1/1     Running   1 (88s ago)     2m12s
kube-system   pod/coredns-5966c6bdcd-2gxvq                        1/1     Running   0               4m2s
kube-system   pod/coredns-5966c6bdcd-qfmn2                        1/1     Running   0               4m2s
kube-system   pod/kube-apiserver-alt-orchestra-27m-03d            1/1     Running   0               3m49s
kube-system   pod/kube-apiserver-alt-orchestra-gpg-qkl            1/1     Running   0               3m55s
kube-system   pod/kube-apiserver-alt-orchestra-urk-gud            1/1     Running   0               3m18s
kube-system   pod/kube-controller-manager-alt-orchestra-27m-03d   1/1     Running   0               3m49s
kube-system   pod/kube-controller-manager-alt-orchestra-gpg-qkl   1/1     Running   1 (115s ago)    3m55s
kube-system   pod/kube-controller-manager-alt-orchestra-urk-gud   1/1     Running   2 (4m22s ago)   3m18s
kube-system   pod/kube-scheduler-alt-orchestra-27m-03d            1/1     Running   0               3m49s
kube-system   pod/kube-scheduler-alt-orchestra-gpg-qkl            1/1     Running   0               3m55s
kube-system   pod/kube-scheduler-alt-orchestra-urk-gud            1/1     Running   4 (77s ago)     3m17s

NAMESPACE     NAME                   TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                  AGE
default       service/kubernetes     ClusterIP   10.96.0.1      <none>        443/TCP                  5m32s
kube-system   service/cilium-envoy   ClusterIP   None           <none>        9964/TCP                 2m12s
kube-system   service/hubble-peer    ClusterIP   10.110.143.8   <none>        443/TCP                  2m12s
kube-system   service/kube-dns       ClusterIP   10.96.0.10     <none>        53/UDP,53/TCP,9153/TCP   4m16s

NAMESPACE     NAME                          DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
kube-system   daemonset.apps/cilium         5         5         5       5            5           kubernetes.io/os=linux   2m12s
kube-system   daemonset.apps/cilium-envoy   5         5         5       5            5           kubernetes.io/os=linux   2m12s

NAMESPACE     NAME                              READY   UP-TO-DATE   AVAILABLE   AGE
kube-system   deployment.apps/cilium-operator   2/2     2            2           2m12s
kube-system   deployment.apps/coredns           2/2     2            2           4m16s

NAMESPACE     NAME                                         DESIRED   CURRENT   READY   AGE
kube-system   replicaset.apps/cilium-operator-67f4b4f5fd   2         2         2       2m12s
kube-system   replicaset.apps/coredns-5966c6bdcd           2         2         2       4m2s

Получить список сервисов (services) во всех пространствах имён (namespaces) в кластере Kubernetes:

console
$ kubectl get svc -A

Примерный вывод:

text
NAMESPACE     NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE
default       kubernetes     ClusterIP   10.96.0.1      <none>        443/TCP                  6m2s
kube-system   cilium-envoy   ClusterIP   None           <none>        9964/TCP                 2m42s
kube-system   hubble-peer    ClusterIP   10.110.143.8   <none>        443/TCP                  2m42s
kube-system   kube-dns       ClusterIP   10.96.0.10     <none>        53/UDP,53/TCP,9153/TCP   4m46s

Изменить конфигурацию endpoint и node:

console
$ talosctl config endpoint 192.168.1.2 192.168.1.3 192.168.1.4
$ talosctl config node 192.168.1.2 192.168.1.3 192.168.1.4 192.168.1.5 192.168.1.6

Вывести информацию о сервисах, работающих на узлах ALT Orchestra, включая их имена, состоянию и конфигурацию.

console
$ talosctl -n 192.168.1.2 service
$ talosctl -n 192.168.1.3 service
$ talosctl -n 192.168.1.4 service
$ talosctl -n 192.168.1.5 service
$ talosctl -n 192.168.1.6 service

Примерный вывод для Controlplane:

text
NODE          SERVICE      STATE     HEALTH   LAST CHANGE   LAST EVENT
192.168.1.4   apid         Running   OK       5m41s ago     Health check successful
192.168.1.4   auditd       Running   OK       5m52s ago     Health check successful
192.168.1.4   containerd   Running   OK       5m52s ago     Health check successful
192.168.1.4   cri          Running   OK       5m41s ago     Health check successful
192.168.1.4   dashboard    Running   ?        5m42s ago     Process Process(["/sbin/dashboard"]) started with PID 2204
192.168.1.4   etcd         Running   OK       3m37s ago     Health check successful
192.168.1.4   kubelet      Running   OK       5m35s ago     Health check successful
192.168.1.4   machined     Running   OK       5m52s ago     Health check successful
192.168.1.4   syslogd      Running   OK       5m51s ago     Health check successful
192.168.1.4   trustd       Running   OK       5m41s ago     Health check successful
192.168.1.4   udevd        Running   OK       5m43s ago     Health check successful

Примерный вывод для Worker:

text
NODE          SERVICE      STATE     HEALTH   LAST CHANGE   LAST EVENT
192.168.1.5   apid         Running   OK       5m40s ago     Health check successful
192.168.1.5   auditd       Running   OK       5m48s ago     Health check successful
192.168.1.5   containerd   Running   OK       5m48s ago     Health check successful
192.168.1.5   cri          Running   OK       5m39s ago     Health check successful
192.168.1.5   dashboard    Running   ?        5m41s ago     Process Process(["/sbin/dashboard"]) started with PID 2094
192.168.1.5   kubelet      Running   OK       5m34s ago     Health check successful
192.168.1.5   machined     Running   OK       5m48s ago     Health check successful
192.168.1.5   syslogd      Running   OK       5m47s ago     Health check successful
192.168.1.5   udevd        Running   OK       5m42s ago     Health check successful

Получить информацию о членах кластера ALT Orchestra:

console
talosctl get members -n 192.168.1.2 -e 192.168.1.2
talosctl get members -n 192.168.1.3 -e 192.168.1.3
talosctl get members -n 192.168.1.4 -e 192.168.1.4
talosctl get members -n 192.168.1.5 -e 192.168.1.5
talosctl get members -n 192.168.1.6 -e 192.168.1.6

Присутствуют все узлы кластера:

text
NODE          NAMESPACE   TYPE     ID                      VERSION   HOSTNAME                MACHINE TYPE   OS                        ADDRESSES
192.168.1.2   cluster     Member   alt-orchestra-4nl-pux   1         alt-orchestra-4nl-pux   controlplane   ALT Orchestra (v11.0)   ["192.168.1.2"]
192.168.1.2   cluster     Member   alt-orchestra-8vo-85l   1         alt-orchestra-8vo-85l   controlplane   ALT Orchestra (v11.0)   ["192.168.1.3"]
192.168.1.2   cluster     Member   alt-orchestra-hku-1uk   1         alt-orchestra-hku-1uk   worker         ALT Orchestra (v11.0)   ["192.168.1.5"]
192.168.1.2   cluster     Member   alt-orchestra-tab-x0d   1         alt-orchestra-tab-x0d   worker         ALT Orchestra (v11.0)   ["192.168.1.6"]
192.168.1.2   cluster     Member   alt-orchestra-y3q-ik2   1         alt-orchestra-y3q-ik2   controlplane   ALT Orchestra (v11.0)   ["192.168.1.4"]

Выполнить healthcheck для узлов Controlplane:

console
$ talosctl health -n 192.168.1.2
$ talosctl health -n 192.168.1.3
$ talosctl health -n 192.168.1.4

Примерный вывод:

text
discovered nodes: ["192.168.1.2" "192.168.1.3" "192.168.1.4" "192.168.1.5" "192.168.1.6"]
waiting for etcd to be healthy: ...
waiting for etcd to be healthy: OK
waiting for etcd members to be consistent across nodes: ...
waiting for etcd members to be consistent across nodes: OK
waiting for etcd members to be control plane nodes: ...
waiting for etcd members to be control plane nodes: OK
waiting for apid to be ready: ...
waiting for apid to be ready: OK
waiting for all nodes memory sizes: ...
waiting for all nodes memory sizes: OK
waiting for all nodes disk sizes: ...
waiting for all nodes disk sizes: OK
waiting for no diagnostics: ...
waiting for no diagnostics: OK
waiting for kubelet to be healthy: ...
waiting for kubelet to be healthy: OK
waiting for all nodes to finish boot sequence: ...
waiting for all nodes to finish boot sequence: OK
waiting for all k8s nodes to report: ...
waiting for all k8s nodes to report: OK
waiting for all control plane static pods to be running: ...
waiting for all control plane static pods to be running: OK
waiting for all control plane components to be ready: ...
waiting for all control plane components to be ready: OK
waiting for all k8s nodes to report ready: ...
waiting for all k8s nodes to report ready: OK
waiting for coredns to report ready: ...
waiting for coredns to report ready: OK
waiting for all k8s nodes to report schedulable: ...
waiting for all k8s nodes to report schedulable: OK

Ручное обновление кластера ​

Получить текущую версию кластера:

console
$ kubectl get nodes -o jsonpath='{.items[0].status.nodeInfo.kubeletVersion}'

Вывод:

text
v1.32.7

Обновить кэш:

console
REPO=p11 && \
KUBELET_VERSION="v1.33.3" && \
KUBELETIMAGE="registry.altlinux.org/${REPO}/kubelet:${KUBELET_VERSION}" && \
APISERVERIMAGE="registry.altlinux.org/${REPO}/kube-apiserver:${KUBELET_VERSION}" && \
CONTROLMANAGERIMAGE="registry.altlinux.org/${REPO}/kube-controller-manager:${KUBELET_VERSION}" && \
SHEDULERIMAGE="registry.altlinux.org/${REPO}/kube-scheduler:${KUBELET_VERSION}"

for image in ${KUBELETIMAGE} ${APISERVERIMAGE} ${CONTROLMANAGERIMAGE} ${SHEDULERIMAGE} ${COREDNSIMAGE} ${ETCDIMAGE} ${PAUSEIMAGE}
do
    podman pull --tls-verify=false $image
done

Проверить их наличие в кэше:

console
$ podman images

Обновить кластер Kubernetes до версии 1.33.3:

console
$ talosctl --nodes 192.168.1.2 upgrade-k8s --to 1.33.3

Примерно обновление каждой машины:

text
. . . . .
 > "192.168.1.4": starting update
 > update kube-scheduler: v1.32.7 -> 1.33.3
 > "192.168.1.4": machine configuration patched
 > "192.168.1.4": waiting for kube-scheduler pod update
 > "192.168.1.4": kube-scheduler: waiting, config version mismatch: got "1", expected "2"
 > "192.168.1.4": kube-scheduler: waiting, config version mismatch: got "1", expected "2"
 > "192.168.1.4": kube-scheduler: waiting, config version mismatch: got "1", expected "2"
 > "192.168.1.4": kube-scheduler: waiting, config version mismatch: got "1", expected "2"
 > "192.168.1.4": kube-scheduler: pod is not ready, waiting
 > "192.168.1.4": kube-scheduler: pod is not ready, waiting
 > "192.168.1.4": kube-scheduler: pod is not ready, waiting
 < "192.168.1.4": successfully updated
. . . . .

Кластер успешно обновился:

text
. . . . .
 > processing manifest v1.Secret/kube-system/bootstrap-token-u8gr7g
 < no changes
 > processing manifest rbac.authorization.k8s.io/v1.ClusterRoleBinding/system-bootstrap-approve-node-client-csr
 < no changes
 > processing manifest rbac.authorization.k8s.io/v1.ClusterRoleBinding/system-bootstrap-node-bootstrapper
 < no changes
 > processing manifest rbac.authorization.k8s.io/v1.ClusterRoleBinding/system-bootstrap-node-renewal
 < no changes
. . . . .
 < applied successfully
 > processing manifest v1.ServiceAccount/kube-system/coredns
 < no changes
 > processing manifest rbac.authorization.k8s.io/v1.ClusterRoleBinding/system:coredns
 < no changes
 > processing manifest rbac.authorization.k8s.io/v1.ClusterRole/system:coredns
 < no changes
 > processing manifest v1.ConfigMap/kube-system/coredns
 < no changes
 > processing manifest apps/v1.Deployment/kube-system/coredns
 < no changes
 > processing manifest v1.Service/kube-system/kube-dns
 < no changes
 > processing manifest v1.ConfigMap/kube-system/kubeconfig-in-cluster
 < no changes
waiting for all manifests to be applied

Получить текущую версию кластера:

console
$ kubectl get nodes -o jsonpath='{.items[0].status.nodeInfo.kubeletVersion}'

Вывод:

text
v1.33.3

Убедиться, что все ноды обновлены до версии v1.33.3:

text
# kubectl get nodes
NAME                    STATUS   ROLES           AGE    VERSION
alt-orchestra-4nl-pux   Ready    control-plane   139m   v1.33.3
alt-orchestra-8vo-85l   Ready    control-plane   139m   v1.33.3
alt-orchestra-hku-1uk   Ready    <none>          139m   v1.33.3
alt-orchestra-tab-x0d   Ready    <none>          139m   v1.33.3
alt-orchestra-y3q-ik2   Ready    control-plane   139m   v1.33.3

Вывести список всех ресурсов во всех пространствах имён кластера Kubernetes:

console
$ kubectl get all -A

Все поды в состоянии Running:

text
NAMESPACE     NAME                                                READY   STATUS    RESTARTS      AGE
kube-system   pod/cilium-2jvb4                                    1/1     Running   0             32m
kube-system   pod/cilium-4xrlg                                    1/1     Running   1 (31m ago)   32m
kube-system   pod/cilium-9nspx                                    1/1     Running   0             32m
kube-system   pod/cilium-9zb6v                                    1/1     Running   1 (24m ago)   32m
kube-system   pod/cilium-cp96k                                    1/1     Running   0             32m
kube-system   pod/cilium-envoy-2m8wp                              1/1     Running   0             32m
kube-system   pod/cilium-envoy-4mx5h                              1/1     Running   0             32m
kube-system   pod/cilium-envoy-g8qh5                              1/1     Running   0             32m
kube-system   pod/cilium-envoy-xffdn                              1/1     Running   0             32m
kube-system   pod/cilium-envoy-z8xw6                              1/1     Running   1 (24m ago)   32m
kube-system   pod/cilium-operator-67f4b4f5fd-9djtg                1/1     Running   3 (16m ago)   32m
kube-system   pod/cilium-operator-67f4b4f5fd-wkjdq                1/1     Running   2 (17m ago)   32m
kube-system   pod/coredns-5966c6bdcd-2gxvq                        1/1     Running   0             34m
kube-system   pod/coredns-5966c6bdcd-qfmn2                        1/1     Running   0             34m
kube-system   pod/kube-apiserver-alt-orchestra-27m-03d            1/1     Running   1 (17m ago)   17m
kube-system   pod/kube-apiserver-alt-orchestra-gpg-qkl            1/1     Running   0             16m
kube-system   pod/kube-apiserver-alt-orchestra-urk-gud            1/1     Running   1 (16m ago)   16m
kube-system   pod/kube-controller-manager-alt-orchestra-27m-03d   1/1     Running   2 (17m ago)   17m
kube-system   pod/kube-controller-manager-alt-orchestra-gpg-qkl   1/1     Running   0             16m
kube-system   pod/kube-controller-manager-alt-orchestra-urk-gud   1/1     Running   1 (16m ago)   16m
kube-system   pod/kube-scheduler-alt-orchestra-27m-03d            1/1     Running   2 (17m ago)   17m
kube-system   pod/kube-scheduler-alt-orchestra-gpg-qkl            1/1     Running   0             16m
kube-system   pod/kube-scheduler-alt-orchestra-urk-gud            1/1     Running   1 (16m ago)   16m

NAMESPACE     NAME                   TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                  AGE
default       service/kubernetes     ClusterIP   10.96.0.1      <none>        443/TCP                  35m
kube-system   service/cilium-envoy   ClusterIP   None           <none>        9964/TCP                 32m
kube-system   service/hubble-peer    ClusterIP   10.110.143.8   <none>        443/TCP                  32m
kube-system   service/kube-dns       ClusterIP   10.96.0.10     <none>        53/UDP,53/TCP,9153/TCP   34m

NAMESPACE     NAME                          DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
kube-system   daemonset.apps/cilium         5         5         5       5            5           kubernetes.io/os=linux   32m
kube-system   daemonset.apps/cilium-envoy   5         5         5       5            5           kubernetes.io/os=linux   32m

NAMESPACE     NAME                              READY   UP-TO-DATE   AVAILABLE   AGE
kube-system   deployment.apps/cilium-operator   2/2     2            2           32m
kube-system   deployment.apps/coredns           2/2     2            2           34m

NAMESPACE     NAME                                         DESIRED   CURRENT   READY   AGE
kube-system   replicaset.apps/cilium-operator-67f4b4f5fd   2         2         2       32m
kube-system   replicaset.apps/coredns-5966c6bdcd           2         2         2       34m

Выполнить healhcheck для узлов Controlplane:

console
$ talosctl health -n 192.168.1.2
$ talosctl health -n 192.168.1.3
$ talosctl health -n 192.168.1.4

Healthcheck выполнен успешно:

text
discovered nodes: ["192.168.1.2" "192.168.1.3" "192.168.1.4" "192.168.1.5" "192.168.1.6"]
waiting for etcd to be healthy: ...
waiting for etcd to be healthy: OK
waiting for etcd members to be consistent across nodes: ...
waiting for etcd members to be consistent across nodes: OK
waiting for etcd members to be control plane nodes: ...
waiting for etcd members to be control plane nodes: OK
waiting for apid to be ready: ...
waiting for apid to be ready: OK
waiting for all nodes memory sizes: ...
waiting for all nodes memory sizes: OK
waiting for all nodes disk sizes: ...
waiting for all nodes disk sizes: OK
waiting for no diagnostics: ...
waiting for no diagnostics: OK
waiting for kubelet to be healthy: ...
waiting for kubelet to be healthy: OK
waiting for all nodes to finish boot sequence: ...
waiting for all nodes to finish boot sequence: OK
waiting for all k8s nodes to report: ...
waiting for all k8s nodes to report: OK
waiting for all control plane static pods to be running: ...
waiting for all control plane static pods to be running: OK
waiting for all control plane components to be ready: ...
waiting for all control plane components to be ready: OK
waiting for all k8s nodes to report ready: ...
waiting for all k8s nodes to report ready: OK
waiting for coredns to report ready: ...
waiting for coredns to report ready: OK
waiting for all k8s nodes to report schedulable: ...
waiting for all k8s nodes to report schedulable: OK

Опубликовано под лицензией GPL-3.0+. Содержание доступно по лицензии CC BY-SA 4.0, если не указано иное. Разработано участниками ALT Orchestra.