ingress 简介 Ingress 公开了从集群外部到集群内服务的 HTTP 和 HTTPS 路由。 流量路由由 Ingress 资源上定义的规则控制。
1 2 3 4 5 internet | [ Ingress ] --|-----|-- [ Services ]
可以将 Ingress 配置为服务提供外部可访问的 URL、负载均衡流量、终止 SSL/TLS，以及提供基于名称的虚拟主机等能力。 Ingress 控制器 通常负责通过负载均衡器来实现 Ingress，尽管它也可以配置边缘路由器或其他前端来帮助处理流量。
ingress控制器借助service的服务发现机制实现配置的动态更新以实现Pod的负载均衡机制实现，由于涉及到Ingress Controller的动态更新，目前社区Ingress Controller大体包含两种类型的控制器：
传统的七层负载均衡如Nginx，HAproxy，开发了适应微服务应用的插件，具有成熟，高性能等优点；
类型
常见类型
优点
缺点
传统负载均衡
nginx，haproxy
成熟，稳定，高性能
动态更新需reload配置文件
微服务负载均衡
Traefik，Envoy，Istio
天生为微服务而生，动态更新
性能还有待提升
安装 使用 NodePort的方式部署 ingress-nginx-controller
1 2 3 wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.35.0/deploy/static/provider/baremetal/deploy.yaml mv deploy.yaml ingress-deploy-hostnetwork.yaml kubectl apply -f ingress-deploy-hostnetwork.yaml
检查服务启动是否正常(拉镜像需要等待一段时间)
1 kubectl get pods -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx
检查服务是否正常
1 2 3 4 5 6 7 8 9 10 11 12 POD_NAMESPACE=ingress-nginx POD_NAME=$(kubectl get pods -n $POD_NAMESPACE -l app.kubernetes.io/name=ingress-nginx --field-selector=status.phase=Running -o jsonpath='{.items[0].metadata.name}') kubectl exec -it $POD_NAME -n $POD_NAMESPACE -- /nginx-ingress-controller --version ------------------------------------------------------------------------------- NGINX Ingress controller Release: v0.35.0 Build: 54ad65e32bcab32791ab18531a838d1c0f0811ef Repository: https://github.com/kubernetes/ingress-nginx nginx version: nginx/1.19.2 -------------------------------------------------------------------------------
配置本地网络启动 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 apiVersion: apps/v1 kind: DaemonSet metadata: labels: ..... name: ingress-nginx-controller namespace: ingress-nginx spec: selector: matchLabels: ..... revisionHistoryLimit: 10 minReadySeconds: 0 template: metadata: labels: ..... spec: hostNetwork: true dnsPolicy: ClusterFirstWithHostNet
修改内容:
kind: DaemonSet 配置服务启动方式为DaemonSet(原来是deployment)。配置 spec.hostNetwork: true,本地网络启动。
配置 dns的策略为: ClusterFirstWithHostNet, 与hostNetwork 配合使用，参考官方文档pod-s-dns-policy
重新同步一下pod文件
1 kubectl apply -f ingress-deploy-hostnetwork.yaml
配置标签，让ingress 只调度到Master节点上 为了让网关的性能更好，我们需要给ingress打标签，让ingress调度到master机器上。同时给master 打上污点，禁止其他pod调度到master节点上。
1 2 3 4 5 6 7 8 9 kubectl get pods -n ingress-nginx -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES ingress-nginx-admission-create-z7sts 0/1 Completed 0 2d16h 172.30.199.211 node7 <none> <none> ingress-nginx-admission-patch-4qgtz 0/1 Completed 0 2d16h 172.30.139.14 node6 <none> <none> ingress-nginx-controller-h5fdd 1/1 Running 1 2d16h 192.168.66.16 node6 <none> <none> ingress-nginx-controller-j6n9z 1/1 Running 0 2d16h 192.168.66.11 node1 <none> <none> ingress-nginx-controller-kfnxt 1/1 Running 1 2d16h 192.168.66.17 node7 <none> <none> ingress-nginx-controller-n5pl4 1/1 Running 0 2d16h 192.168.66.12 node2 <none> <none> ingress-nginx-controller-qzbb5 1/1 Running 0 2d16h 192.168.66.13 node3 <none> <none>
对 node1 node2 打标签
1 2 kubectl label nodes node1 api/ingress-controller=true kubectl label nodes node2 api/ingress-controller=true
查看标签
1 2 3 4 5 6 7 8 # 为了好看，去掉相同的标签 kubectl get nodes --show-labels NAME STATUS ROLES AGE VERSION LABELS node1 Ready <none> 3h53m v1.16.6 api/ingress-controller=true,kubernetes.io/hostname=node1 node2 Ready <none> 3h53m v1.16.6 api/ingress-controller=true,kubernetes.io/hostname=node2 node3 Ready <none> 3h53m v1.16.6 kubernetes.io/hostname=node3 node6 Ready <none> 3d4h v1.16.6 kubernetes.io/hostname=node6 node7 Ready <none> 3d4h v1.16.6 kubernetes.io/hostname=node7
增加标签选择，及容忍污点
1 2 3 4 5 6 7 8 9 10 11 # vim ingress-deploy-hostnetwork.yaml spec: hostNetwork: true dnsPolicy: ClusterFirstWithHostNet nodeSelector: api/ingress-controller: "true" tolerations: - key: "node-role.kubernetes.io/master" operator: "Equal" value: "" effect: "NoSchedule"
在刚才修改网络的后面继续增加以上命令
从新同步一下配置
1 kubectl apply -f ingress-deploy-hostnetwork.yaml
1 2 3 4 5 6 [root@node1 ~]# kubectl get pods -n ingress-nginx -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES ingress-nginx-admission-create-z7sts 0/1 Completed 0 2d16h 172.30.199.211 node7 <none> <none> ingress-nginx-admission-patch-4qgtz 0/1 Completed 0 2d16h 172.30.139.14 node6 <none> <none> ingress-nginx-controller-6s2lp 1/1 Running 0 57m 192.168.66.12 node2 <none> <none> ingress-nginx-controller-j6n9z 1/1 Running 0 57m 192.168.66.11 node1 <none> <none>
为了避免master节点会被调度新的pod，需要给master节点打上Taints。
1 2 kubectl taint nodes node1 node-role.kubernetes.io/master=:NoSchedule kubectl taint nodes node2 node-role.kubernetes.io/master=:NoSchedule
1 2 3 4 5 [root@node1 ~]# kubectl describe node node1 |grep Taints Taints: node-role.kubernetes.io/master:NoSchedule [root@node1 ~]# kubectl describe node node2 |grep Taints Taints: node-role.kubernetes.io/master:NoSchedule [root@node1 ~]#
1 2 3 4 5 6 7 [root@node1 ~]# kubectl get pods -n ingress-nginx -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES ingress-nginx-admission-create-z7sts 0/1 Completed 0 2d19h 172.30.199.211 node7 <none> <none> ingress-nginx-admission-patch-4qgtz 0/1 Completed 0 2d19h 172.30.139.14 node6 <none> <none> ingress-nginx-controller-6s2lp 1/1 Running 0 3h6m 192.168.66.12 node2 <none> <none> ingress-nginx-controller-kv4xs 1/1 Running 0 3h5m 192.168.66.11 node1 <none> <none> [root@node1 ~]#
汇总一下配置文件的修改内容
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 apiVersion: apps/v1 kind: DaemonSet metadata: labels: ..... name: ingress-nginx-controller namespace: ingress-nginx spec: selector: matchLabels: ..... revisionHistoryLimit: 10 minReadySeconds: 0 template: metadata: labels: ..... spec: hostNetwork: true dnsPolicy: ClusterFirstWithHostNet nodeSelector: api/ingress-controller: "true" tolerations: - key: "node-role.kubernetes.io/master" operator: "Equal" value: "" effect: "NoSchedule"
修改参数如下：
kind: Deployment #修改为DaemonSet
hostNetwork: true #添加该字段让docker使用物理机网络，在物理机暴露服务端口(80)，注意物理机80端口提前不能被占用
dnsPolicy: ClusterFirstWithHostNet #使用hostNetwork后容器会使用物理机网络包括DNS，会无法解析内部service，使用此参数让容器使用K8S的DNS
nodeSelector:api/ingress-controller: “true” #添加节点标签
tolerations: 添加对指定节点容忍度
验证ingress 是否正常 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 cat > nginx-test.yaml <<EOF # deploy apiVersion: apps/v1 kind: Deployment metadata: name: nginx-test spec: selector: matchLabels: app: nginx-test replicas: 1 template: metadata: labels: app: nginx-test spec: containers: - name: nginx-test image: nginx ports: - containerPort: 80 --- # service apiVersion: v1 kind: Service metadata: name: nginx-test spec: ports: - port: 80 protocol: TCP targetPort: 80 selector: app: nginx-test --- # ingress apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: nginx-test spec: rules: - host: nginx.biglittleant.cn http: paths: - path: / backend: serviceName: nginx-test servicePort: 80 EOF
1 apply -f nginx-test.yaml
检查执行结果
1 2 3 4 5 6 7 8 9 10 [root@node1 ~]# kubectl get pods NAME READY STATUS RESTARTS AGE nginx-test-54789bbd4-tplgz 1/1 Running 0 2m27s [root@node1 ~]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.254.0.1 <none> 443/TCP 18h nginx-test ClusterIP 10.254.51.46 <none> 80/TCP 3m57s [root@node1 ~]# kubectl get ingress NAME HOSTS ADDRESS PORTS AGE nginx-test nginx.biglittleant.cn 192.168.66.11,192.168.66.12 80 4m2s
本地访问确认ingress工作正常
1 2 3 4 5 6 7 8 9 10 11 curl -I nginx.biglittleant.cn -x 192.168.66.11:80 HTTP/1.1 200 OK Server: nginx/1.19.2 Date: Wed, 30 Sep 2020 03:39:53 GMT Content-Type: text/html Content-Length: 612 Connection: keep-alive Vary: Accept-Encoding Last-Modified: Tue, 11 Aug 2020 14:50:35 GMT ETag: "5f32b03b-264" Accept-Ranges: bytes
在NGINX容器中新建一个文件，确认访问是否正常。
1 2 3 4 5 6 7 [root@node1 ~]# kubectl exec -it nginx-test-54789bbd4-tplgz /bin/bash root@nginx-test-54789bbd4-tplgz:/# echo 1 > /usr/share/nginx/html/1.html root@nginx-test-54789bbd4-tplgz:/# cat /usr/share/nginx/html/1.html 1 root@nginx-test-54789bbd4-tplgz:/# exit exit [root@node1 ~]#
1 2 curl nginx.biglittleant.cn/1.html -x 192.168.66.11:80 1
参考文章 官方仓库 官方文档 k8s配置ingress nginx ingress的安装配置和使用