安装

环境 arm平台

Kernel

Linux rock64 4.4.132-1075-rockchip-ayufan-ga83beded8524 #1 SMP Thu Jul 26 08:22:22 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux

System

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.4 LTS
Release:        18.04
Codename:       bionic

Memory
              total        used        free      shared  buff/cache   available
Mem:           3924         198         105           3        3619        3692
Swap:          1962           0        1961

Network

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:cf:ca:aa:ce  txqueuelen 0  (Ethernet)
        RX packets 43147  bytes 2477474 (2.4 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 43353  bytes 92327114 (92.3 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.58  netmask 255.255.255.0  broadcast 192.168.1.255
        ether 06:14:d5:2c:82:aa  txqueuelen 1000  (Ethernet)
        RX packets 2303981  bytes 1891341723 (1.8 GB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1427379  bytes 369022653 (369.0 MB)
        TX errors 2  dropped 0 overruns 2  carrier 0  collisions 0
        device interrupt 40

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        loop  txqueuelen 1  (Local Loopback)
        RX packets 1514  bytes 138738 (138.7 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1514  bytes 138738 (138.7 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Disk
Filesystem      Size  Used Avail Use% Mounted on
udev            2.0G     0  2.0G   0% /dev
tmpfs           393M  664K  392M   1% /run
/dev/mmcblk1p7  118G   47G   66G  42% /
.....

禁用swap

sudo apt install dphys-swapfile \
sudo dphys-swapfile swapoff && \
sudo dphys-swapfile uninstall && \
sudo update-rc.d dphys-swapfile remove

检查交换禁用是否成功

sudo swapon --summary

docker cgroup使用systemd

见问题一

安装kubelet kubectl kubeadm

增加阿里源 /etc/apt/sources.list

deb https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial main

执行

curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -
apt-get update
apt-get install -y kubelet kubeadm kubectl

下载对应的镜像

1.查看当前kubeadm对应的镜像

kubeadm config images list

# version detail:

k8s.gcr.io/kube-apiserver:v1.18.3
k8s.gcr.io/kube-controller-manager:v1.18.3
k8s.gcr.io/kube-scheduler:v1.18.3
k8s.gcr.io/kube-proxy:v1.18.3
k8s.gcr.io/pause:3.2
k8s.gcr.io/etcd:3.4.3-0
k8s.gcr.io/coredns:1.6.7

2.下载 kube-* image

设置对应的版本,里面有这kube-*四个镜像

wget https://dl.k8s.io/v1.18.3/kubernetes-server-linux-arm64.tar.gz

3.使用tar命令解压

4.快速导入

ls -al kubernetes/server/bin/*.tar | awk '{print "docker load -i "$NF}' | bash

5.处理被墙的pause、etcd、coredns

登录国外vps, 下载对应平台的image,并推动到hub.docker.com上。绕一圈。

docker pull k8s.gcr.io/pause-arm64:3.2
docker pull k8s.gcr.io/etcd-arm64:3.4.3-0 

docker tag k8s.gcr.io/pause-arm64:3.2 clibing/k8s-pause-arm64:3.2 
docker tag k8s.gcr.io/etcd-arm64:3.4.3-0 clibing/k8s-etcd-arm64:3.4.3-0 

docker push clibing/k8s-pause-arm64:3.2 
docker push clibing/k8s-etcd-arm64:3.4.3-0  

coredns

6.在arm主机上执行

docker pull coredns/coredns:1.6.7
docker tag coredns/coredns:1.6.7 k8s.gcr.io/coredns:1.6.7

7.验证是否一致

kubeadm init

kubeadm init --kubernetes-version=1.18.3  \
--apiserver-advertise-address=192.168.1.58   \
--service-cidr=10.10.0.0/16 --pod-network-cidr=10.122.0.0/16

问题一

  • [WARNING IsDockerSystemdCheck]: detected “cgroupfs” as the Docker cgroup driver. The recommended driver is “systemd”. Please follow the guide at https://kubernetes.io/docs/setup/cri/

vi /etc/docker/daemon.json,加入下述内容:

{
  "exec-opts": ["native.cgroupdriver=systemd"]
}

重启docker:

systemctl restart docker

查看修改后的状态:

docker info | grep Cgroup
  • [ERROR Swap]: running with swap on is not supported. Please disable swap
sync; echo 3 > /proc/sys/vm/drop_caches
swapoff -a

已经执行swap了, 不知道为什么没有生效,估计是没有重启rock64

问题二

  • join node token expire

默认token的有效期为24小时,当过期之后,该token就不可用了。解决方法如下:

  • 重新生成新的token
kubeadm token create
kubeadm token list

TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS aa78f6.8b4cafc8ed26c34f 23h 2017-12-26T16:36:29+08:00 authentication,signing system:bootstrappers:kubeadm:default-node-token

  • 获取ca证书sha256编码hash值
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
  • 节点加入集群
kubeadm join 192.168.1.58:6443 --token qz7h0p.bn867v3gbsksg2p9 \
    --discovery-token-ca-cert-hash sha256:计算出来的值

参考