环境说明

官方说明：

下载所需的 yaml 文件

mkdir ~/ingress && cd ~/ingress
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.45.0/deploy/static/provider/baremetal/deploy.yaml	

修改配置文件

这里演示的是高可用的 nginx-ingress-controller 服务。

# 在 ingress-nginx-controller 容器的 deploy.spec 添加 replicas: 2
spec:
  replicas: 2

将原本的 nodeport 修改成 clusterIP

# 在 ingress-nginx-controller service的 svc.spec 注释掉 type: NodePort
spec:
  # type: NodePort

将容器端口映射到宿主机

# 在 ingress-nginx-controller 容器的 deploy.spec.template.spec 添加 hostNetwork: true
    spec:
      hostNetwork: true

# 在 ingress-nginx-controller 容器的 deploy.spec.template.spec.containers.ports 添加 hostPost 字段
          ports:
            - name: http
              containerPort: 80 # 添加的字段
              hostPort: 80
              protocol: TCP
            - name: https
              containerPort: 443 # 添加的字段
              hostPort: 443
              protocol: TCP
            - name: webhook
              containerPort: 8443
              protocol: TCP

修改DNS的策略

# 在 ingress-nginx-controller 容器的 deploy.spec.template.spec 修改 dnsPolicy
    spec:
      dnsPolicy: ClusterFirstWithHostNet

修改下载镜像路径

# 在 ingress-nginx-controller 容器的 deploy.spec.template.spec.containers 修改 image 字段
      containers:
        - name: controller
          image: registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v0.45.0

指定 pod 调度特定节点

# 节点添加标签
kubectl label node k8s-node02 kubernetes.io/ingress=nginx
kubectl label node k8s-node03 kubernetes.io/ingress=nginx

# 在 ingress-nginx-controller 容器的 deploy.spec.template.spec 修改 nodeSelector
      nodeSelector:
        kubernetes.io/ingress: nginx

# 在 ingress-nginx-controller 容器的 deploy.spec.template.spec 添加 affinity
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchLabels:
                  app.kubernetes.io/name: ingress-nginx
              topologyKey: kubernetes.io/hostname

启动服务

$ kubectl apply -f deploy.yaml 
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
service/ingress-nginx-controller-admission created
service/ingress-nginx-controller created
deployment.apps/ingress-nginx-controller created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
serviceaccount/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created

$ kubectl -n ingress-nginx get pod -owide
NAME                                        READY   STATUS      RESTARTS   AGE   IP               NODE           NOMINATED NODE   READINESS GATES
ingress-nginx-admission-create-tm6hb        0/1     Completed   0          21s   20.0.85.198      k8s-node01                
ingress-nginx-admission-patch-64bgc         0/1     Completed   1          21s   20.0.32.136      k8s-master01              
ingress-nginx-controller-656cf6c7fd-lw9dx   1/1     Running     0          21s   192.168.32.138   k8s-node03                
ingress-nginx-controller-656cf6c7fd-ncsrz   1/1     Running     0          21s   192.168.32.137   k8s-node02                

验证

创建nginx应用

cat > nginx.yaml <<-EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-nginx
spec:
  selector:
    matchLabels:
      app: my-nginx
  template:
    metadata:
      labels:
        app: my-nginx
    spec:
      containers:
      - name: my-nginx
        image: nginx
        resources:
          limits:
            memory: "200Mi"
            cpu: "500m"
          requests:
            memory: "100Mi"
            cpu: "100m"
        ports:
        - name: web
          containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: my-nginx
  ports:
  - port: 80
    targetPort: web
EOF

$ kubectl apply -f nginx.yaml 
deployment.apps/my-nginx created
service/nginx-service created

$ kubectl get pod -owide
NAME                        READY   STATUS    RESTARTS   AGE     IP            NODE         NOMINATED NODE   READINESS GATES
my-nginx-759cf4d696-vkj4q   1/1     Running   0          4m10s   20.0.85.199   k8s-node01              

创建ingress资源

$ cat > nginx-ingress.yaml <<-EOF
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ingress
  labels:
      name: nginx-ingress
spec:
  backend:
    serviceName: nginx-service
    servicePort: 80
  rules:
  - host: www.ecloud.com
    http:
      paths:
      - path: /
        backend:
          serviceName: nginx-service
          servicePort: 80
EOF

$  kubectl apply -f nginx-ingress.yaml 
ingress.extensions/nginx-ingress created

$ kubectl get ingress
NAME            CLASS    HOSTS            ADDRESS                         PORTS   AGE
nginx-ingress      www.ecloud.com   192.168.32.137,192.168.32.138   80      21s

使用域名访问

$ echo '192.168.32.137 www.ecloud.com' >> /etc/hosts

$ curl www.ecloud.com







Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.

For online documentation and support please refer to
nginx.org.

Commercial support is available at
nginx.com.

Thank you for using nginx.

可以通过 keepalived + LVS 高可用，使用 VIP 做域名解析。这里就不实现了。

sysctl 调优

# 临时临时
kubectl patch deployment -n ingress-nginx nginx-ingress-controller \
    --patch="$(curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/docs/examples/customization/sysctl/patch.json)"

# 永久生效
# 在 ingress-nginx-controller 容器的 deploy.spec.template.spec 添加 initContainers
      initContainers:
      - name: sysctl
        image: alpine:3.13
        securityContext:
          privileged: true
        command: ["sh", "-c", "sysctl -w net.core.somaxconn=32768; sysctl -w net.ipv4.ip_local_port_range='32768 65535'"]

变化：

• 积压队列设置net.core.somaxconn从128到32768
• 临时端口设置net.ipv4.ip_local_port_range从32768 60999到32768 65535(符合端口规划)

附加iptables规则

iptables -t filter -I INPUT -p tcp -m multiport --dport 80,443,8443 -m comment --comment "nginx ingress controller ports" -j ACCEPT