안녕하세요.
'수상한 김토끼' 입니다.
이번 장에서는 컨테이너 환경을 오케스트레이션 하기 위해 가장 많이 쓰이는 k8s 환경을 VM으로 로컬에 구성해 보고 알아가는 시간을 갖도록 하겠습니다.
이번 장 진행에 참고한 k8s 공식 문서입니다.
https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/
Installing kubeadm
This page shows how to install the kubeadm toolbox. For information on how to create a cluster with kubeadm once you have performed this installation process, see the Creating a cluster with kubeadm page. Before you begin A compatible Linux host. The Kuber
kubernetes.io
진행에 문제가 있거나 추가적인 내용은 위 문서를 참조해 주세요.
이 블로그 글은 미들웨어 경험이 없으신 분들도 쉽게 따라 하실 수 있도록 쉽게 작성하는 것이 목표입니다.
설명을 보고 진행하시다가 궁금하신 내용은 댓글로 문의하시면 가능한 범위 내에서 알려 드리도록 하겠습니다.
1. VM 설치
k8s 환경을 구성하기 위해 가장 먼저 Linux VM을 설치해야 합니다.
저는 Ubuntu 22.04 LTS 버전을 사용할 예정임으로 Ubuntu 공식 홈페이지에서 설치파일을 다운로드 받아 줍니다.
https://ubuntu.com/download/server
Get Ubuntu Server | Download | Ubuntu
Get Ubuntu Server one of three ways; by using Multipass on your desktop, using MAAS to provision machines in your data centre or installing it directly on a server.
ubuntu.com
Server 버전의 설치 이미지 파일을 다운로드 받아 줍니다.
VM 하이퍼바이저는 Windows 10에서 제공되는 Hyper-V를 사용하도록 하겠습니다.
Hyper-V 설치 방법은 여기에서는 다루지 않을 예정이니 인터넷 검색등을 통해 확인하시거나 다른 하이퍼바이저 또는 클라우드 서비스를 사용하실수도 있으니 편하신 방법을 선택해 주세요.
Hyper-V를 실행하면 다음과 같은 화면을 보실 수 있으며, k8s 구성 및 실습을 위해서 총 3개의 VM을 생성할 예정입니다.
우선 k8s-master 라는 이름의 vm을 생성 해 보겠습니다.
VM을 생성하기 위해 오른쪽 작업 tab의 새로 만들기를 눌러 관련 내용을 입력 해 줍니다.
아래와 같이 k8s-master vm을 생성합니다.
생성된 vm을 실행 후 Ubuntu 설치를 진행합니다.
설치과정을 자세히 다루지는 않겠습니다.
다음과 같이 설치를 진행하여 vm 3대를 생성 해 줍니다.
VM 3개(k8s-master, k8s-node1, k8s-node2)생성이 완료되었습니다.
2. Docker 설치
생성한 VM 3대에 k8s 관련 도구를 설치하고 환경구성을 진행해 보겠습니다.
우선 Docker를 VM 3대 모두에 설치 해 줍니다. 포스팅은 k8s-master 서버에서만 수행하지만 3개 서버 모두 설치를 진행해 주세요.
docker 공식 홈페이지에 내용대로 설치를 진행하겠습니다.
https://docs.docker.com/engine/install/ubuntu/
Install Docker Engine on Ubuntu
docs.docker.com
첫번째로 아래 명령을 수행해 줍니다. (Update the apt package index and install packages to allow apt to use a repository over HTTPS)
sudo apt-get update
sudo apt-get install \
ca-certificates \
curl \
gnupg수행 결과입니다.
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
|
kami@k8s-master:~$ sudo apt-get update
Hit:1 http://kr.archive.ubuntu.com/ubuntu focal InRelease
Get:2 http://kr.archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:3 http://kr.archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB]
Get:4 http://kr.archive.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Get:5 http://kr.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,465 kB]
Fetched 2,801 kB in 14s (194 kB/s)
Reading package lists... Done
kami@k8s-master:~$ sudo apt-get install \
> ca-certificates \
> curl \
> gnupg
Reading package lists... Done
Building dependency tree
Reading state information... Done
ca-certificates is already the newest version (20211016ubuntu0.20.04.1).
ca-certificates set to manually installed.
curl is already the newest version (7.68.0-1ubuntu2.18).
curl set to manually installed.
gnupg is already the newest version (2.2.19-3ubuntu2.2).
gnupg set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 10 not upgraded.
kami@k8s-master:~$
|
cs |
두번째 진행 내용입니다. (Add Docker’s official GPG key)
sudo mkdir -m 0755 -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg|
1
2
3
|
kami@k8s-master:~$ sudo mkdir -m 0755 -p /etc/apt/keyrings
kami@k8s-master:~$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
kami@k8s-master:~$
|
cs |
세번째 진행 내용입니다. (Use the following command to set up the repository)
echo \
"deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
"$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null|
1
2
3
4
5
|
kami@k8s-master:~$ echo \
> "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
> "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
> sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
kami@k8s-master:~$
|
cs |
네 번째로 apt 저장소를 업데이트 해 줍니다. (Install Docker Engine, containerd, and Docker Compose)
sudo apt-get update|
1
2
3
4
5
6
7
8
9
10
|
kami@k8s-master:~$ sudo apt-get update
Get:1 https://download.docker.com/linux/ubuntu focal InRelease [57.7 kB]
Get:2 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages [26.8 kB]
Hit:3 http://kr.archive.ubuntu.com/ubuntu focal InRelease
Hit:4 http://kr.archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:5 http://kr.archive.ubuntu.com/ubuntu focal-backports InRelease
Hit:6 http://kr.archive.ubuntu.com/ubuntu focal-security InRelease
Fetched 84.5 kB in 1s (71.4 kB/s)
Reading package lists... Done
kami@k8s-master:~$
|
cs |
다섯번 째로 docker를 설치 해 줍니다. (To install the latest version, run)
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
|
kami@k8s-master:~$ sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
docker-ce-rootless-extras pigz slirp4netns
Suggested packages:
aufs-tools cgroupfs-mount | cgroup-lite
The following NEW packages will be installed:
containerd.io docker-buildx-plugin docker-ce docker-ce-cli docker-ce-rootless-extras docker-compose-plugin pigz slirp4netns
0 upgraded, 8 newly installed, 0 to remove and 10 not upgraded.
Need to get 109 MB of archives.
After this operation, 395 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 https://download.docker.com/linux/ubuntu focal/stable amd64 containerd.io amd64 1.6.20-1 [28.3 MB]
Get:2 http://kr.archive.ubuntu.com/ubuntu focal/universe amd64 pigz amd64 2.4-1 [57.4 kB]
Get:3 http://kr.archive.ubuntu.com/ubuntu focal/universe amd64 slirp4netns amd64 0.4.3-1 [74.3 kB]
Get:4 https://download.docker.com/linux/ubuntu focal/stable amd64 docker-buildx-plugin amd64 0.10.4-1~ubuntu.20.04~focal [25.9 MB]
Get:5 https://download.docker.com/linux/ubuntu focal/stable amd64 docker-ce-cli amd64 5:23.0.3-1~ubuntu.20.04~focal [13.2 MB]
Get:6 https://download.docker.com/linux/ubuntu focal/stable amd64 docker-ce amd64 5:23.0.3-1~ubuntu.20.04~focal [22.0 MB]
Get:7 https://download.docker.com/linux/ubuntu focal/stable amd64 docker-ce-rootless-extras amd64 5:23.0.3-1~ubuntu.20.04~focal [8,778 kB]
Get:8 https://download.docker.com/linux/ubuntu focal/stable amd64 docker-compose-plugin amd64 2.17.2-1~ubuntu.20.04~focal [10.9 MB]
Fetched 109 MB in 10s (10.6 MB/s)
Selecting previously unselected package pigz.
(Reading database ... 72298 files and directories currently installed.)
Preparing to unpack .../0-pigz_2.4-1_amd64.deb ...
Unpacking pigz (2.4-1) ...
Selecting previously unselected package containerd.io.
Preparing to unpack .../1-containerd.io_1.6.20-1_amd64.deb ...
Unpacking containerd.io (1.6.20-1) ...
Selecting previously unselected package docker-buildx-plugin.
Preparing to unpack .../2-docker-buildx-plugin_0.10.4-1~ubuntu.20.04~focal_amd64.deb ...
Unpacking docker-buildx-plugin (0.10.4-1~ubuntu.20.04~focal) ...
Selecting previously unselected package docker-ce-cli.
Preparing to unpack .../3-docker-ce-cli_5%3a23.0.3-1~ubuntu.20.04~focal_amd64.deb ...
Unpacking docker-ce-cli (5:23.0.3-1~ubuntu.20.04~focal) ...
Selecting previously unselected package docker-ce.
Preparing to unpack .../4-docker-ce_5%3a23.0.3-1~ubuntu.20.04~focal_amd64.deb ...
Unpacking docker-ce (5:23.0.3-1~ubuntu.20.04~focal) ...
Selecting previously unselected package docker-ce-rootless-extras.
Preparing to unpack .../5-docker-ce-rootless-extras_5%3a23.0.3-1~ubuntu.20.04~focal_amd64.deb ...
Unpacking docker-ce-rootless-extras (5:23.0.3-1~ubuntu.20.04~focal) ...
Selecting previously unselected package docker-compose-plugin.
Preparing to unpack .../6-docker-compose-plugin_2.17.2-1~ubuntu.20.04~focal_amd64.deb ...
Unpacking docker-compose-plugin (2.17.2-1~ubuntu.20.04~focal) ...
Selecting previously unselected package slirp4netns.
Preparing to unpack .../7-slirp4netns_0.4.3-1_amd64.deb ...
Unpacking slirp4netns (0.4.3-1) ...
Setting up slirp4netns (0.4.3-1) ...
Setting up docker-buildx-plugin (0.10.4-1~ubuntu.20.04~focal) ...
Setting up containerd.io (1.6.20-1) ...
Created symlink /etc/systemd/system/multi-user.target.wants/containerd.service → /lib/systemd/system/containerd.service.
Setting up docker-compose-plugin (2.17.2-1~ubuntu.20.04~focal) ...
Setting up docker-ce-cli (5:23.0.3-1~ubuntu.20.04~focal) ...
Setting up pigz (2.4-1) ...
Setting up docker-ce-rootless-extras (5:23.0.3-1~ubuntu.20.04~focal) ...
Setting up docker-ce (5:23.0.3-1~ubuntu.20.04~focal) ...
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /lib/systemd/system/docker.service.
Created symlink /etc/systemd/system/sockets.target.wants/docker.socket → /lib/systemd/system/docker.socket.
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for systemd (245.4-4ubuntu3.20) ...
kami@k8s-master:~$
|
cs |
docker 설치가 완료 되었습니다.
master, node1, node2 모두 docker 설치가 되어야 k8s 구성이 가능하다는 점을 잊지 마시고 VM 3대 모두 docker를 설치 해 주세요.
3. k8s 설치
k8s(kubelet, kubeadm and kubectl)을 VM 3대 모두 설치 해 줍니다.
k8s(kubelet, kubeadm and kubectl)은 k8s에서 공식 제공하는 클러스터 생성/관리 도구로 자세한 내용은 공식 홈페이지를 참조 해 주세요.
https://kubernetes.io/ko/docs/setup/production-environment/tools/kubeadm/install-kubeadm/
kubeadm 설치하기
이 페이지에서는 kubeadm 툴박스 설치 방법을 보여준다. 이 설치 프로세스를 수행한 후 kubeadm으로 클러스터를 만드는 방법에 대한 자세한 내용은 kubeadm으로 클러스터 생성하기 페이지를 참고한다.
kubernetes.io
k8s(kubelet, kubeadm and kubectl) 설치 전에 ubuntu의 swap을 disable 해주어야 합니다.
보다 상세한 내용은 아래 페이지를 참조해 주세요.
https://graspingtech.com/disable-swap-ubuntu/
How to Disable Swap on Ubuntu Linux 20.04 - GraspingTech
10 Mar 2021 (Updated 8 May 2021)
graspingtech.com
ubuntu swap을 disable하기 위해서는 다음 3단계의 절차가 필요합니다.
sudo swapoff -asudo rm /swap.img/etc/fstab에서 다음 라인을 지워주세요.
/swap.img none swap sw 0 0swap을 disable 하신 후 서버의 방화벽도 내린 후 다음 단계를 진행 해 주세요.
운영환경에서는 k8s가 사용하는 포트만을 오픈해야 되겠지만 본 포스팅에서는 편의상 방화벽을 내린 후 진행하겠습니다.
Ubuntu 방화벽 해제는 ufw 명령어를 사용합니다.
sudo service ufw stop
swap과 방화벽 해제까지 마무리 되었으니 k8s(kubelet, kubeadm and kubectl)를 설치 해 보겠습니다.
VM 3대 모두에서 아래 설치과정을 진행해 주세요.
첫번째는 k8s 저장소를 사용하기 위한 apt 설정입니다. (Update the apt package index and install packages needed to use the Kubernetes apt repository)
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
|
kami@k8s-master:~$ sudo apt-get update
Hit:1 https://download.docker.com/linux/ubuntu focal InRelease
Hit:2 http://kr.archive.ubuntu.com/ubuntu focal InRelease
Hit:3 http://kr.archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:4 http://kr.archive.ubuntu.com/ubuntu focal-backports InRelease
Hit:5 http://kr.archive.ubuntu.com/ubuntu focal-security InRelease
Reading package lists... Done
kami@k8s-master:~$ sudo apt-get install -y apt-transport-https ca-certificates curl
Reading package lists... Done
Building dependency tree
Reading state information... Done
ca-certificates is already the newest version (20211016ubuntu0.20.04.1).
curl is already the newest version (7.68.0-1ubuntu2.18).
The following NEW packages will be installed:
apt-transport-https
0 upgraded, 1 newly installed, 0 to remove and 10 not upgraded.
Need to get 1,704 B of archives.
After this operation, 162 kB of additional disk space will be used.
Get:1 http://kr.archive.ubuntu.com/ubuntu focal-updates/universe amd64 apt-transport-https all 2.0.9 [1,704 B]
Fetched 1,704 B in 1s (3,311 B/s)
Selecting previously unselected package apt-transport-https.
(Reading database ... 72554 files and directories currently installed.)
Preparing to unpack .../apt-transport-https_2.0.9_all.deb ...
Unpacking apt-transport-https (2.0.9) ...
Setting up apt-transport-https (2.0.9) ...
kami@k8s-master:~$
|
cs |
두번째는 구글 public 사인키 다운로드 입니다. (Download the Google Cloud public signing key)
sudo curl -fsSLo /etc/apt/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg|
1
2
|
kami@k8s-master:~$ sudo curl -fsSLo /etc/apt/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg
kami@k8s-master:~$
|
cs |
세번째로 k8s apt 저장소를 등록 합니다. (Add the Kubernetes apt repository)
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list|
1
2
3
|
kami@k8s-master:~$ echo "deb [signed-by=/etc/apt/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb [signed-by=/etc/apt/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main
kami@k8s-master:~$
|
cs |
마지막으로 apt 저장소를 업데이트하고 k8s를 설치합니다. (Update apt package index, install kubelet, kubeadm and kubectl, and pin their version)
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
sudo systemctl start kubelet
sudo systemctl enable kubelet|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
|
kami@k8s-master:~$ sudo apt-get update
Hit:2 https://download.docker.com/linux/ubuntu focal InRelease
Hit:3 http://kr.archive.ubuntu.com/ubuntu focal InRelease
Hit:4 http://kr.archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:5 http://kr.archive.ubuntu.com/ubuntu focal-backports InRelease
Get:1 https://packages.cloud.google.com/apt kubernetes-xenial InRelease [8,993 B]
Hit:6 http://kr.archive.ubuntu.com/ubuntu focal-security InRelease
Get:7 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 Packages [64.5 kB]
Fetched 73.5 kB in 2s (39.4 kB/s)
Reading package lists... Done
kami@k8s-master:~$ sudo apt-get install -y kubelet kubeadm kubectl
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
conntrack cri-tools ebtables kubernetes-cni socat
Suggested packages:
nftables
The following NEW packages will be installed:
conntrack cri-tools ebtables kubeadm kubectl kubelet kubernetes-cni socat
0 upgraded, 8 newly installed, 0 to remove and 10 not upgraded.
Need to get 87.3 MB of archives.
After this operation, 341 MB of additional disk space will be used.
Get:2 http://kr.archive.ubuntu.com/ubuntu focal/main amd64 conntrack amd64 1:1.4.5-2 [30.3 kB]
Get:1 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 cri-tools amd64 1.26.0-00 [18.9 MB]
Get:6 http://kr.archive.ubuntu.com/ubuntu focal/main amd64 ebtables amd64 2.0.11-3build1 [80.3 kB]
Get:8 http://kr.archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB]
Get:3 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubernetes-cni amd64 1.2.0-00 [27.6 MB]
Get:4 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubelet amd64 1.26.3-00 [20.5 MB]
Get:5 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubectl amd64 1.26.3-00 [10.1 MB]
Get:7 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubeadm amd64 1.26.3-00 [9,747 kB]
Fetched 87.3 MB in 10s (8,876 kB/s)
Selecting previously unselected package conntrack.
(Reading database ... 72558 files and directories currently installed.)
Preparing to unpack .../0-conntrack_1%3a1.4.5-2_amd64.deb ...
Unpacking conntrack (1:1.4.5-2) ...
Selecting previously unselected package cri-tools.
Preparing to unpack .../1-cri-tools_1.26.0-00_amd64.deb ...
Unpacking cri-tools (1.26.0-00) ...
Selecting previously unselected package ebtables.
Preparing to unpack .../2-ebtables_2.0.11-3build1_amd64.deb ...
Unpacking ebtables (2.0.11-3build1) ...
Selecting previously unselected package kubernetes-cni.
Preparing to unpack .../3-kubernetes-cni_1.2.0-00_amd64.deb ...
Unpacking kubernetes-cni (1.2.0-00) ...
Selecting previously unselected package socat.
Preparing to unpack .../4-socat_1.7.3.3-2_amd64.deb ...
Unpacking socat (1.7.3.3-2) ...
Selecting previously unselected package kubelet.
Preparing to unpack .../5-kubelet_1.26.3-00_amd64.deb ...
Unpacking kubelet (1.26.3-00) ...
Selecting previously unselected package kubectl.
Preparing to unpack .../6-kubectl_1.26.3-00_amd64.deb ...
Unpacking kubectl (1.26.3-00) ...
Selecting previously unselected package kubeadm.
Preparing to unpack .../7-kubeadm_1.26.3-00_amd64.deb ...
Unpacking kubeadm (1.26.3-00) ...
Setting up conntrack (1:1.4.5-2) ...
Setting up kubectl (1.26.3-00) ...
Setting up ebtables (2.0.11-3build1) ...
Setting up socat (1.7.3.3-2) ...
Setting up cri-tools (1.26.0-00) ...
Setting up kubernetes-cni (1.2.0-00) ...
Setting up kubelet (1.26.3-00) ...
Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /lib/systemd/system/kubelet.service.
Setting up kubeadm (1.26.3-00) ...
Processing triggers for man-db (2.9.1-1) ...
kami@k8s-master:~$
|
cs |
k8s(kubelet, kubeadm and kubectl) 설치가 완료 되었습니다.
k8s 싱글 노드 멀티 마스터 구성을 위해 'Creating a cluster with kubeadm' 문서를 참조하여 다음 작업을 진행합니다.
https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/
Creating a cluster with kubeadm
Using kubeadm, you can create a minimum viable Kubernetes cluster that conforms to best practices. In fact, you can use kubeadm to set up a cluster that will pass the Kubernetes Conformance tests. kubeadm also supports other cluster lifecycle functions, su
kubernetes.io
이 작업은 master 노드에서만 실행 해 주셔야 합니다. node1, node2에서는 수행하시면 안 됩니다.
sudo rm /etc/containerd/config.toml
sudo systemctl restart containerd
sudo kubeadm init|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
|
kami@k8s-master:~$ sudo rm /etc/containerd/config.toml
kami@k8s-master:~$ sudo systemctl restart containerd
kami@k8s-master:~$ sudo kubeadm init
[init] Using Kubernetes version: v1.26.3
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [k8s-master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.0.79]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [k8s-master localhost] and IPs [192.168.0.79 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [k8s-master localhost] and IPs [192.168.0.79 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[apiclient] All control plane components are healthy after 49.502851 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node k8s-master as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node k8s-master as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]
[bootstrap-token] Using token: pn115z.frjj24uy5tmiybrm
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.0.79:6443 --token pn115z.frjj24uy5tmiybrm \
--discovery-token-ca-cert-hash sha256:de63c200bd30599459d7c617e63cb6a218e630ceefbc5b49086a574583838f44
kami@k8s-master:~$
|
cs |
마지막에 표시된 kubeadm join 문구는 node1, node2에서 master노드에 접근할 때 사용되어야 함으로 별도로 기록하여 보관 해 주세요.
추가로 kubectl 명령을 실행할 수 있는 환경을 만들기 위해서 kubectl을 사용할 유저로 접속하여 60번째 줄부터 62번째 줄까지의 명령을 복사하여 실행 후 kubectl 명령을 수행해 봅니다.
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config|
1
2
3
4
5
6
7
|
root@k8s-master:/home/kami# mkdir -p $HOME/.kube
root@k8s-master:/home/kami# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
root@k8s-master:/home/kami# sudo chown $(id -u):$(id -g) $HOME/.kube/config
root@k8s-master:/home/kami# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master NotReady control-plane 5m3s v1.26.3
root@k8s-master:/home/kami#
|
cs |
kubectl이 정상적으로 수행되었지만 CNI가 아직 설치되지 않아서 NotReady 상태로 확인 됩니다.
다음으로 CNI(VxLab or Pod Network)를 설치 해 줍니다. 본 예제에서는 CNI의 한 종류인 weave net을 설치하겠습니다.
https://kubernetes.io/ko/docs/tasks/administer-cluster/network-policy-provider/weave-network-policy/
네트워크 폴리시로 위브넷(Weave Net) 사용하기
이 페이지는 네트워크 폴리시(NetworkPolicy)로 위브넷(Weave Net)를 사용하는 방법을 살펴본다. 시작하기 전에 쿠버네티스 클러스터가 필요하다. 맨 땅에서부터 시작하기를 위해서 kubeadm 시작하기 안
kubernetes.io
설치 가이드가 나와 있는 weave net 공식문서를 참조하여 설치를 진행해 줍니다.
https://www.weave.works/docs/net/latest/kubernetes/kube-addon/
Integrating Kubernetes via the Addon
The following topics are discussed: Installation Before installing Weave Net, you should make sure the following ports are not blocked by your firewall: TCP 6783 and UDP 6783/6784. For more details, see the FAQ. Weave Net can be installed onto your CNI-ena
www.weave.works
이 작업도 master 노드에서만 실행해야 함을 주의 해 주세요.
kubectl apply -f https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s.yaml|
1
2
3
4
5
6
7
8
9
10
11
|
root@k8s-master:/home/kami# kubectl apply -f https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s.yaml
serviceaccount/weave-net created
clusterrole.rbac.authorization.k8s.io/weave-net created
clusterrolebinding.rbac.authorization.k8s.io/weave-net created
role.rbac.authorization.k8s.io/weave-net created
rolebinding.rbac.authorization.k8s.io/weave-net created
daemonset.apps/weave-net created
root@k8s-master:/home/kami# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready control-plane 13m v1.26.3
root@k8s-master:/home/kami#
|
cs |
k8s 구성이 완료되었습니다.
4. k8s worker 노드 구성
Master 노드 구성이 완료되었으니 worker 노드를 master 노드에 join 시켜 보겠습니다.
k8s 클러스터 구성 시 출력 되었던 join 문구를 node1, node2에서 실행해 주면 join은 완료 됩니다.
sudo rm /etc/containerd/config.toml
sudo systemctl restart containerd
kubeadm join 192.168.0.79:6443 --token pn115z.frjj24uy5tmiybrm \
--discovery-token-ca-cert-hash sha256:de63c200bd30599459d7c617e63cb6a218e630ceefbc5b49086a574583838f44|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
|
root@k8s-node2:~# sudo rm /etc/containerd/config.toml
root@k8s-node2:~# sudo systemctl restart containerd
root@k8s-node2:~# kubeadm join 192.168.0.79:6443 --token pn115z.frjj24uy5tmiybrm --discovery-token-ca-cert-hash sha256:de63c200bd30599459d7c617e63cb6a218e630ceefbc5b49086a574583838f44
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
root@k8s-node2:~#
|
cs |
정상적으로 join이 완료 되었으니 master 노드에서 k8s 클러스터 상태를 마무리 해 보고 이번 포스팅은 마치도록 하겠습니다.
|
1
2
3
4
5
6
|
root@k8s-master:/home/kami# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k8s-master Ready control-plane 37m v1.26.3 192.168.0.79 <none> Ubuntu 20.04.6 LTS 5.4.0-146-generic containerd://1.6.20
k8s-node1 Ready <none> 3m20s v1.26.3 192.168.0.77 <none> Ubuntu 20.04.6 LTS 5.4.0-146-generic containerd://1.6.20
k8s-node2 Ready <none> 3m v1.26.3 192.168.0.78 <none> Ubuntu 20.04.6 LTS 5.4.0-146-generic containerd://1.6.20
root@k8s-master:/home/kami#
|
cs |
여기까지 VM 3대를 생성하여 k8s환경 구성이 완료되었습니다. 그대로 따라하시면 큰 어려움 없이 k8s환경 구성이 가능할 것으로 보입니다.
k8s구성은 여기에서 마무리 하고 k8s의 api에 대한 자세한 내용은 다음 포스팅에서 다루도록 하겠습니다.
'developerworks > K8s' 카테고리의 다른 글
| 2장. k8s (kubeadm 클러스터) 업그레이드 (0) | 2024.03.05 |
|---|---|
| 1-1장. k8s 저장소 변경내역 업데이트 (0) | 2024.03.05 |