kubernetes k8s ingress

搭建Kubernetes集群(2) - Ingress

Posted by Vincent on Thursday, April 4, 2019

Ingress

Ingress允许你配置运行于K8S上的HTTP负载均衡器对服务进行负载均衡. Ingress支持以下功能

  • 基于内容的路由
    • 基于主机名的路由
    • 基于路径的路由
  • 支持TLS/SSL请求

查看官网对Ingress简介

Ingress 控制器

Ingress控制器是运行在集群中根据Ingress资源进行配置的HTTP负载均衡器应用.这个均衡器可以是运行在集群上的软件负载均衡也可以是运行在外部的硬件设备.不同的均衡器必须实现不同的Ingress控制器.

对于Nginx而言,Ingress控制器和负载均衡运行于Pod中.

Nginx Ingress控制器

Nginx Ingress控制器支持标准的Ingress功能,基于内容的内容和SSL.

另外可以通过增加 annotation 和 ConfigMap 资源到Ingress资源来适应一些扩展功能.除HTTP外 NGINX Ingress控制器还支持 WebSocket , gRPC,TCP ,UDP 应用的负载均衡(点此查看详细文档).查看文档了解支持的功能和定制的选项.

安装Ingress 控制器

1.创建Namespace,一个 SA ,秘钥和定制的ConfigMap

# ns-and-sa.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: nginx-ingress 
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress 
  namespace: nginx-ingress

kubectl apply -f ns-and-sa.yaml 创建一个命名空间和服务账户.

创建一个秘钥

#default-server-secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: default-server-secret
  namespace: nginx-ingress
type: Opaque
data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN2akNDQWFZQ0NRREFPRjl0THNhWFhEQU5CZ2txaGtpRzl3MEJBUXNGQURBaE1SOHdIUVlEVlFRRERCWk8KUjBsT1dFbHVaM0psYzNORGIyNTBjbTlzYkdWeU1CNFhEVEU0TURreE1qRTRNRE16TlZvWERUSXpNRGt4TVRFNApNRE16TlZvd0lURWZNQjBHQTFVRUF3d1dUa2RKVGxoSmJtZHlaWE56UTI5dWRISnZiR3hsY2pDQ0FTSXdEUVlKCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUwvN2hIUEtFWGRMdjNyaUM3QlBrMTNpWkt5eTlyQ08KR2xZUXYyK2EzUDF0azIrS3YwVGF5aGRCbDRrcnNUcTZzZm8vWUk1Y2Vhbkw4WGM3U1pyQkVRYm9EN2REbWs1Qgo4eDZLS2xHWU5IWlg0Rm5UZ0VPaStlM2ptTFFxRlBSY1kzVnNPazFFeUZBL0JnWlJVbkNHZUtGeERSN0tQdGhyCmtqSXVuektURXUyaDU4Tlp0S21ScUJHdDEwcTNRYzhZT3ExM2FnbmovUWRjc0ZYYTJnMjB1K1lYZDdoZ3krZksKWk4vVUkxQUQ0YzZyM1lma1ZWUmVHd1lxQVp1WXN2V0RKbW1GNWRwdEMzN011cDBPRUxVTExSakZJOTZXNXIwSAo1TmdPc25NWFJNV1hYVlpiNWRxT3R0SmRtS3FhZ25TZ1JQQVpQN2MwQjFQU2FqYzZjNGZRVXpNQ0F3RUFBVEFOCkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWpLb2tRdGRPcEsrTzhibWVPc3lySmdJSXJycVFVY2ZOUitjb0hZVUoKdGhrYnhITFMzR3VBTWI5dm15VExPY2xxeC9aYzJPblEwMEJCLzlTb0swcitFZ1U2UlVrRWtWcitTTFA3NTdUWgozZWI4dmdPdEduMS9ienM3bzNBaS9kclkrcUI5Q2k1S3lPc3FHTG1US2xFaUtOYkcyR1ZyTWxjS0ZYQU80YTY3Cklnc1hzYktNbTQwV1U3cG9mcGltU1ZmaXFSdkV5YmN3N0NYODF6cFErUyt1eHRYK2VBZ3V0NHh3VlI5d2IyVXYKelhuZk9HbWhWNThDd1dIQnNKa0kxNXhaa2VUWXdSN0diaEFMSkZUUkk3dkhvQXprTWIzbjAxQjQyWjNrN3RXNQpJUDFmTlpIOFUvOWxiUHNoT21FRFZkdjF5ZytVRVJxbStGSis2R0oxeFJGcGZnPT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
  tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBdi91RWM4b1JkMHUvZXVJTHNFK1RYZUprckxMMnNJNGFWaEMvYjVyYy9XMlRiNHEvClJOcktGMEdYaVN1eE9ycXgrajlnamx4NXFjdnhkenRKbXNFUkJ1Z1B0ME9hVGtIekhvb3FVWmcwZGxmZ1dkT0EKUTZMNTdlT1l0Q29VOUZ4amRXdzZUVVRJVUQ4R0JsRlNjSVo0b1hFTkhzbysyR3VTTWk2Zk1wTVM3YUhudzFtMApxWkdvRWEzWFNyZEJ6eGc2clhkcUNlUDlCMXl3VmRyYURiUzc1aGQzdUdETDU4cGszOVFqVUFQaHpxdmRoK1JWClZGNGJCaW9CbTVpeTlZTW1hWVhsMm0wTGZzeTZuUTRRdFFzdEdNVWozcGJtdlFmazJBNnljeGRFeFpkZFZsdmwKMm82MjBsMllxcHFDZEtCRThCay90elFIVTlKcU56cHpoOUJUTXdJREFRQUJBb0lCQVFDZklHbXowOHhRVmorNwpLZnZJUXQwQ0YzR2MxNld6eDhVNml4MHg4Mm15d1kxUUNlL3BzWE9LZlRxT1h1SENyUlp5TnUvZ2IvUUQ4bUFOCmxOMjRZTWl0TWRJODg5TEZoTkp3QU5OODJDeTczckM5bzVvUDlkazAvYzRIbjAzSkVYNzZ5QjgzQm9rR1FvYksKMjhMNk0rdHUzUmFqNjd6Vmc2d2szaEhrU0pXSzBwV1YrSjdrUkRWYmhDYUZhNk5nMUZNRWxhTlozVDhhUUtyQgpDUDNDeEFTdjYxWTk5TEI4KzNXWVFIK3NYaTVGM01pYVNBZ1BkQUk3WEh1dXFET1lvMU5PL0JoSGt1aVg2QnRtCnorNTZud2pZMy8yUytSRmNBc3JMTnIwMDJZZi9oY0IraVlDNzVWYmcydVd6WTY3TWdOTGQ5VW9RU3BDRkYrVm4KM0cyUnhybnhBb0dCQU40U3M0ZVlPU2huMVpQQjdhTUZsY0k2RHR2S2ErTGZTTXFyY2pOZjJlSEpZNnhubmxKdgpGenpGL2RiVWVTbWxSekR0WkdlcXZXaHFISy9iTjIyeWJhOU1WMDlRQ0JFTk5jNmtWajJTVHpUWkJVbEx4QzYrCk93Z0wyZHhKendWelU0VC84ajdHalRUN05BZVpFS2FvRHFyRG5BYWkyaW5oZU1JVWZHRXFGKzJyQW9HQkFOMVAKK0tZL0lsS3RWRzRKSklQNzBjUis3RmpyeXJpY05iWCtQVzUvOXFHaWxnY2grZ3l4b25BWlBpd2NpeDN3QVpGdwpaZC96ZFB2aTBkWEppc1BSZjRMazg5b2pCUmpiRmRmc2l5UmJYbyt3TFU4NUhRU2NGMnN5aUFPaTVBRHdVU0FkCm45YWFweUNweEFkREtERHdObit3ZFhtaTZ0OHRpSFRkK3RoVDhkaVpBb0dCQUt6Wis1bG9OOTBtYlF4VVh5YUwKMjFSUm9tMGJjcndsTmVCaWNFSmlzaEhYa2xpSVVxZ3hSZklNM2hhUVRUcklKZENFaHFsV01aV0xPb2I2NTNyZgo3aFlMSXM1ZUtka3o0aFRVdnpldm9TMHVXcm9CV2xOVHlGanIrSWhKZnZUc0hpOGdsU3FkbXgySkJhZUFVWUNXCndNdlQ4NmNLclNyNkQrZG8wS05FZzFsL0FvR0FlMkFVdHVFbFNqLzBmRzgrV3hHc1RFV1JqclRNUzRSUjhRWXQKeXdjdFA4aDZxTGxKUTRCWGxQU05rMXZLTmtOUkxIb2pZT2pCQTViYjhibXNVU1BlV09NNENoaFJ4QnlHbmR2eAphYkJDRkFwY0IvbEg4d1R0alVZYlN5T294ZGt5OEp0ek90ajJhS0FiZHd6NlArWDZDODhjZmxYVFo5MWpYL3RMCjF3TmRKS2tDZ1lCbyt0UzB5TzJ2SWFmK2UwSkN5TGhzVDQ5cTN3Zis2QWVqWGx2WDJ1VnRYejN5QTZnbXo5aCsKcDNlK2JMRUxwb3B0WFhNdUFRR0xhUkcrYlNNcjR5dERYbE5ZSndUeThXczNKY3dlSTdqZVp2b0ZpbmNvVlVIMwphdmxoTUVCRGYxSjltSDB5cDBwWUNaS2ROdHNvZEZtQktzVEtQMjJhTmtsVVhCS3gyZzR6cFE9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=

创建 kubectl apply -f default-server-secret.yaml

默认服务器对没有匹配到Ingress规则的请求返回一个 404 页面. 测试时我们 使用了一个自签名的SSL证书.可以替换成正式的证书

创建用于NGINX的配置

# nginx-config.yaml
kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-config
  namespace: nginx-ingress
data:
  client-max-body-size: "8m"

应用 kubectl apply -f nginx-config.yaml,看看更多选项

2.配置RBAC

如果你的集群启用了RBAC.创建一个集群角色并绑定到上一步创建的服务账户.

#rbac.yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: nginx-ingress
rules:
- apiGroups:
  - ""
  resources:
  - services
  - endpoints
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - get
  - list
  - watch
  - update
  - create
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - list
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - extensions
  resources:
  - ingresses
  verbs:
  - list
  - watch
  - get
- apiGroups:
  - "extensions"
  resources:
  - ingresses/status
  verbs:
  - update
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: nginx-ingress
subjects:
- kind: ServiceAccount
  name: nginx-ingress
  namespace: nginx-ingress
roleRef:
  kind: ClusterRole
  name: nginx-ingress
  apiGroup: rbac.authorization.k8s.io

kubectl apply -f rbac.yaml

执行以上步骤你需要是管理员账户

3.部署Ingress控制器

我们有两种方式来部署Ingress控制器.

  • Deployment.如果你希望动态改变Ingress控制器的实例数量,选用该模式
  • DaementSet.该模式在每个节点和子节部署Ingress控制器

创建Deployment

#nginx-ingress.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-ingress
  namespace: nginx-ingress
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-ingress
  template:
    metadata:
      labels:
        app: nginx-ingress
     #annotations:
       #prometheus.io/scrape: "true"
       #prometheus.io/port: "9113"
    spec:
      serviceAccountName: nginx-ingress
      containers:
      - image: nginx/nginx-ingress:edge
        imagePullPolicy: Always
        name: nginx-ingress
        ports:
        - name: http
          containerPort: 80
        - name: https
          containerPort: 443
       #- name: prometheus
         #containerPort: 9113
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        args:
          - -nginx-configmaps=$(POD_NAMESPACE)/nginx-config
          - -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret
         #- -v=3 # Enables extensive logging. Useful for troubleshooting.
         #- -report-ingress-status
         #- -external-service=nginx-ingress
         #- -enable-leader-election
         #- -enable-prometheus-metrics

kubectl apply -f nginx-ingress.yaml

执行后K8S 将会创建一个Ingress 控制器的POD.

创建DaemonSet

#nginx-ingress.yaml
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: nginx-ingress
  namespace: nginx-ingress
spec:
  selector:
    matchLabels:
      app: nginx-ingress
  template:
    metadata:
      labels:
        app: nginx-ingress
     #annotations:
       #prometheus.io/scrape: "true"
       #prometheus.io/port: "9113"
    spec:
      serviceAccountName: nginx-ingress
      containers:
      - image: nginx/nginx-ingress:edge
        imagePullPolicy: Always
        name: nginx-ingress
        ports:
        - name: http
          containerPort: 80
          hostPort: 80
        - name: https
          containerPort: 443
          hostPort: 443
       #- name: prometheus
         #containerPort: 9113
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        args:
          - -nginx-configmaps=$(POD_NAMESPACE)/nginx-config
          - -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret
         #- -v=3 # Enables extensive logging. Useful for troubleshooting.
         #- -report-ingress-status
         #- -external-service=nginx-ingress
         #- -enable-leader-election
         #- -enable-prometheus-metrics

执行

kubectl apply -f daemon-set/nginx-ingress.yaml

K8S会在集群的每个节点创建POD.查看K8S的相关说明.

kubectl get pods --namespace=nginx-ingress 可以看到每个节点都运行了一个POD .

4 访问Ingress Controller

DaemonSet 模式部署,在每个节点都会部署一个Ingress控制器.所以你可以访问任意一个节点的443和80端口.目前会得到一个404页面.

Deployment模式可以通过暴露NodePort的方式来进行访问.

#nodeport.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx-ingress
  namespace: nginx-ingress
spec:
  type: NodePort 
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
    name: http
  - port: 443
    targetPort: 443
    protocol: TCP
    name: https
  selector:
    app: nginx-ingress

创建一个NodePort类型的服务

 kubectl create -f nodeport.yaml

也可以使用负载均衡服务,参考这里.

5 访问试试监控面板 Stub_status页面

  • Stub_status 默认是开启的.确定nginx-status不要设置false.
  • Stub_status 默认使用8080端口.可以使用 nginx-status-port参数设置.如果你不是8080,在下面的参数下修改.
  • 用kubectl port-forward 功能来访问.
kubectl port-forward <nginx-ingress-pod> 8080:8080 --namespace=nginx-ingress

6 Prometheus 监控支持

你可以使用Promethus收集NGINX的指标.

  • 运行Ingress控制器时,加上 -enable-prometheus-metrics 命令行参数. 这样NGINX将会再通过路径 9113 端口 在/metrics路径暴露Prometheus格式的指标参数.你也可以用个-prometheus-metrics-listen-port修改端口号.
  • 添加Prometheus端口到Ingress控制器的容器.
- name: prometheus
  containerPort: 9113
  • 增加注解到POD模板,让Promethus发现POD
annotations:
    prometheus.io/scrape: "true"
    prometheus.io/port: 9113

7 卸载

直接删除命名空间即可

kubectl delete namespace nginx-ingress

部署和配置应用程序的负载均衡

1 .创建development和Service

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: echo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: echo
  template:
    metadata:
      labels:
        app: echo
    spec:
      containers:
      - name: echo
        image: vincentmi/ng
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: echo-svc
spec:
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
    name: http
  selector:
    app: echo

2.建立Ingress资源

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: echo-ingress
spec:
  rules:
  - host: echo.local.vnzmi.com
    http:
      paths:
      - backend:
          serviceName: echo-svc
          servicePort: 80

修改HOST 之后进行访问即可看到页面了.

/img/in-post/ingress.png

应用部署模板

---
# developement
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: philo-screening-svc
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: philo-screening-svc
        domain: com.philo.office.screening
    spec:
      containers:
        - name: philo-screening-svc
          image: vincentmi/screening
          ports:
            - containerPort: 80

---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: philo-screening-svc
    domain: com.philo.office.screening
  name: philo-screening-svc
  namespace: default
spec:
  ports:
    - port: 80
      targetPort: 80
      name: http
  selector:
    app: philo-screening-svc

---
#--- ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: philo-screening-svc-ingress
spec:
  rules:
    - host: screening.philowork.com
      http:
        paths:
          - backend:
              serviceName: philo-screening-svc
              servicePort: 80

「真诚赞赏，手留余香」

我的乐与怒

真诚赞赏，手留余香

使用微信扫描二维码完成支付

目录
    标签
    aframe andrew-ng arduino blockchain cgi consul docker eth ethereum evm feign flask golang hw java javascript jpa js k8s kubernetes linux mac machine-learning math mysql nginx op opentcv ops poco pptp python rd spring springboot ssl thymeleaf type vagrant vpn vue webrtc workflow 技术文章 新浪博客 无聊 生活 生活记录 股票 运维
    链接