Centos7 安装k8s 1.25.0

• Centos7 安装k8s 1.25.0
  • 安装包
  • 安装containerd
  • 安装阿里云镜像
  • master
  • 安装网络插件

安装包

链接: https://pan.baidu.com/s/1H3M8fo3hh_qj4bJFifLRIQ 密码: psha

安装containerd

yum install ntpdate -y
ntpdate ntp.aliyun.com
# 修改你的网卡ip设置为静态(我这里是eth0)你们的自己看
vim /etc/sysconfig/network-scripts/ifcfg-eth0
service network restart

将BOOTPROTO参数设置为none，并添加好自己的IP与网关地址，网关地址请根据自己的虚拟机为主。
注意设置DNS，我这里没设置不然拉不了镜像，coredns起来不了。

# Generated by parse-kickstart
DEVICE="eth0"
IPV6INIT="yes"
BOOTPROTO="none"
UUID="cd2966b9-8eb4-42bc-8cf5-ec145c343ce9"
ONBOOT="yes"
IPADDR="10.211.55.7"
PREFIX="24"
GATEWAY="10.211.55.1"
DNS1=114.114.114.114
DNS2=8.8.8.8

ls
clear
pwd
df -h
mkdir /k8s
cd /k8s/
# 将我的所有包放到这个目录下
cp containerd.service /etc/systemd/system/containerd.service
clear
ls
tar Cxzvf /usr/local containerd-1.6.9-linux-amd64.tar.gz
cat /etc/systemd/system/containerd.service
clear
systemctl enable --now containerd
install -m 755 runc.amd64 /usr/local/sbin/runc
tar zxvf libseccomp-2.5.4.tar.gz
cd libseccomp-2.5.4/
yum install gperf -y
gpg --verify file.asc file
./configure
make
make install
cd ..
chmod +x runc.amd64
install -m 755 runc.amd64 /usr/local/sbin/runc
runc -v
mkdir -p /home/cni-plugins
tar Cxzvf /home/cni-plugins cni-plugins-linux-amd64-v1.1.1.tgz
mkdir -p /home/cni-tools
mkdir -p /etc/cni/net.d
cp 10-mynet.conf /etc/cni/net.d/10-mynet.conf
cp 99-loopback.conf /etc/cni/net.d/99-loopback.conf
tar Cxzvf /home/cni-tools cni-1.1.2.tar.gz
cd script/
chmod +x ./*.sh
wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
yum -y install jq
sudo CNI_PATH=/home/cni-plugins ./priv-net-run.sh echo "Hello World"

安装阿里云镜像

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
setenforce 0
yum install -y --nogpgcheck kubelet-1.25.3-0 kubeadm-1.25.3-0 kubectl-1.25.3-0
systemctl enable kubelet && systemctl start kubelet
kubelet --version
# 查看需要的镜像
kubeadm config images list

更具不同的节点进行配置.

hostnamectl set-hostname k8s-125-master
hostnamectl set-hostname k8s-125-node1
hostnamectl set-hostname k8s-125-node2

# 关闭防火墙
#关闭防火墙和关闭SELinux
systemctl stop firewalld
systemctl disable firewalld
setenforce 0 # 临时关闭
vim /etc/sysconfig/selinux #永久关闭
# 改为SELINUX=disabled
# 所有节点关闭swap
swapoff -a  #临时关闭
vim /etc/fstab #永久关闭
#注释掉以下字段
# /dev/mapper/cl-swap swap swap defaults 0 0
# 配置内核
cat > kubernetes.conf <<EOF
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
net.ipv4.tcp_tw_recycle=0
vm.swappiness=0
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=1048576
fs.file-max=52706963
fs.nr_open=52706963
net.ipv6.conf.all.disable_ipv6=1
net.netfilter.nf_conntrack_max=2310720
EOF
#将优化内核文件拷贝到/etc/sysctl.d/文件夹下，这样优化文件开机的时候能够被调用
cp kubernetes.conf /etc/sysctl.d/kubernetes.conf
#手动刷新，让优化文件立即生效
modprobe  br_netfilter
modprobe  ip_conntrack
sysctl -p /etc/sysctl.d/kubernetes.conf

# kube-proxy 开启ipvs的前置条件【极其重要】
# 安装ipvsadm工具
yum install ipset ipvsadm -y
cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
EOF
# 添加文件权限
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash
/etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4
# 加载 br_netfilter模块【默认是不加载的，所以需要手动加载】
modprobe br_netfilter
cd /k8s
ctr -n k8s.io images import kube-apiserver-1-25-3.tar
ctr -n k8s.io images import kube-controller-manager-1-25-3.tar
ctr -n k8s.io images import kube-scheduler-1-25-3.tar
ctr -n k8s.io images import kube-proxy-1-25-3.tar
ctr -n k8s.io images import pause-3-8.tar
ctr -n k8s.io images import pause-3-6.tar
ctr -n k8s.io images import etcd-3-5-4-0.tar
ctr -n k8s.io images import coredns-1-9-3.tar
ctr -n k8s.io images ls
crictl images

master

$ kubeadm init --help
# 这样可以
kubeadm init --kubernetes-version=1.25.3 \
--image-repository registry.aliyuncs.com/google_containers \
--apiserver-advertise-address=10.211.55.7 \
--service-cidr=10.96.0.0/12 \
--pod-network-cidr=192.168.0.0/16
# 这样也可以 只是需要修改一下里面的apiserver-advertise-address地址
# 生成文件命令：kubeadm config print init-defaults --kubeconfig ClusterConfiguration > kubeadm.yml
kubeadm init --config=kubeadm.yml --upload-certs | tee kubeadm-init.log
#注意：
# kubernetes-version的值 即为 kubelet --version获得的值
# apiserver-advertise-address这个地址必须是master机器的IP
# 国内可以设置阿里云的--image-repository registry.aliyuncs.com/google_containers \
# 如果有报错，可以执行以下命令进行排查【一般都是因为外网原因拉取不到镜像，可以拉取国内镜像代替】
systemctl status kubelet -l
systemctl status containerd -l
# 查看安装k8s所需的镜像列表
kubeadm config images list
# 解决报错后，要再次运行 kubeadm init，必须首先卸载集群
kubeadm reset
#成功示例：
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
  export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
# 这一行生成的临时令牌[24h过期]是用来加入node节点
kubeadm join 192.168.168.201:6443 --token pq5otc.ker47p9nails0xsf \
        --discovery-token-ca-cert-hash sha256:7a256694edafdbd21b52ca729b0b7ebc142c7fe8435657a6115b95019d2a3178

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config
  export KUBECONFIG=/etc/kubernetes/admin.conf
  # 验证
  kubectl get pods --all-namespaces

安装网络插件

curl https://docs.projectcalico.org/manifests/calico.yaml -O
# 把calico.yaml里pod所在网段改成kubeadm init时选项--pod-network-cidr所指定的网段
# 我们这里默认就是192.168.0.0所以不需要
# sed -i 's/192.168.0.0/10.244.0.0/g' calico.yaml
kubectl apply -f calico.yaml
# 验证
kubectl get pods --all-namespaces

  # 生成新的令牌节点
  kubeadm token create --print-join-command
  # 在其他节点请执行，加入集群中
  kubeadm join 10.211.55.7:6443 --token dhuxe5.oiuoh0ux84dpfs7d --discovery-token-ca-cert-hash sha256:5592e3493fe04e46223023c9232601cb83f3c1d970dd71a3aac357d480087533

启动 IPVS

# 修改mode 为ipvs
kubectl edit -n kube-system cm kube-proxy
# mode: "ipvs"
# 通过下面的命令删除proxy
kubectl get pod -n kube-system |grep kube-proxy |awk '{system("kubectl delete pod "$1" -n kube-system")}'
# 然后我们可以查看proxy 发现ipvs已经启动了
kubectl get pod -n kube-system | grep kube-proxy
kubectl logs -n kube-system kube-proxy-4c5xj
# 也可以这样查看
ipvsadm -Ln

使用crictl发现报错。

crictl ps

原因：未配置endpoints。
指定containerd.sock解决。

crictl config runtime-endpoint unix:///run/containerd/containerd.sock
crictl config image-endpoint unix:///run/containerd/containerd.sock

重启主机后master kubelet报错：unknown service runtime.v1alpha2.RuntimeService
很可能是因为您的 contaienrd 默认配置文件有问题，通常会有一个 disabled_plugins 里面默认配置了 cri。配置文件在 /etc/containerd/config.toml。手动删掉cri；然后重启containerd 即可。

rm /etc/containerd/config.toml 
systemctl restart containerd
vim /etc/containerd/config.toml

disabled_plugins = []
# 同时建议添加 防止 cgroup 问题
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
  [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
    SystemdCgroup = true
# 同时建议添加 pause 镜像配置，国内访问不到 默认的地址
[plugins."io.containerd.grpc.v1.cri"]
  sandbox_image = "k8s.m.daocloud.io/pause:3.7"

systemctl daemon-reload
systemctl restart containerd

发现coredns启动不了是由于/etc/resolv.conf没有配置出现的问题。
可以通过如下方式进行解决。

kubectl edit cm coredns -n kube-system

apiVersion: v1
data:
  Corefile: |
    .:53 {
        errors
        health {
           lameduck 5s
        }
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
           pods insecure
           fallthrough in-addr.arpa ip6.arpa
           ttl 30
        }
        prometheus :9153
        forward . 114.114.114.114 {
           max_concurrent 1000
           policy sequential
        }
        cache 30
        loop
        reload
        loadbalance
    }
kind: ConfigMap
metadata:
  creationTimestamp: "2022-10-27T05:37:37Z"
  name: coredns
  namespace: kube-system
  resourceVersion: "23720"
  uid: 21950342-42d9-4c0e-a610-ce1e5930a8a5

然后再删除coredns便重启成功