{"action":"opened","issue":{"url":"https://api.github.com/repos/kaybinwong/posts/issues/18","repository_url":"https://api.github.com/repos/kaybinwong/posts","labels_url":"https://api.github.com/repos/kaybinwong/posts/issues/18/labels{/name}","comments_url":"https://api.github.com/repos/kaybinwong/posts/issues/18/comments","events_url":"https://api.github.com/repos/kaybinwong/posts/issues/18/events","html_url":"https://github.com/kaybinwong/posts/issues/18","id":310448949,"number":18,"title":"k8s/如何灵活应用Service","user":{"login":"kaybinwong","id":5570216,"avatar_url":"https://avatars0.githubusercontent.com/u/5570216?v=4","gravatar_id":"","url":"https://api.github.com/users/kaybinwong","html_url":"https://github.com/kaybinwong","followers_url":"https://api.github.com/users/kaybinwong/followers","following_url":"https://api.github.com/users/kaybinwong/following{/other_user}","gists_url":"https://api.github.com/users/kaybinwong/gists{/gist_id}","starred_url":"https://api.github.com/users/kaybinwong/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kaybinwong/subscriptions","organizations_url":"https://api.github.com/users/kaybinwong/orgs","repos_url":"https://api.github.com/users/kaybinwong/repos","events_url":"https://api.github.com/users/kaybinwong/events{/privacy}","received_events_url":"https://api.github.com/users/kaybinwong/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-04-02T10:16:04Z","updated_at":"2018-04-02T10:16:04Z","closed_at":null,"author_association":"OWNER","body":"前面我们有过多次创建Pod的示例，但Pod是否能够被直接访问？答案当然是可以的。\r\n首先，我们来定义Deployment。\r\n```\r\n[root@k8s-node1 svr]# cat << EOF > nginx-deployment.yaml\r\napiVersion: apps/v1\r\nkind: Deployment\r\nmetadata:\r\n  name: nginx-deployment\r\n  labels:\r\n    app: nginx\r\nspec:\r\n  replicas: 3\r\n  selector:\r\n    matchLabels:\r\n      app: nginx\r\n  template:\r\n    metadata:\r\n      labels:\r\n        app: nginx\r\n    spec:\r\n      containers:\r\n      - name: nginx\r\n        image: nginx:1.7.9\r\n        ports:\r\n        - containerPort: 80\r\nEOF\r\n```\r\n接着，创建并访问Pod\r\n```\r\n[root@k8s-node1 svr]# kubectl get po -o wide\r\nNAME                                READY     STATUS    RESTARTS   AGE       IP              NODE\r\nnginx-deployment-6c54bd5869-j5njj   1/1       Running   0          16m       10.233.67.31    k8s-node2\r\nnginx-deployment-6c54bd5869-q5hmd   1/1       Running   0          16m       10.233.76.211   k8s-node3\r\nnginx-deployment-6c54bd5869-xxgvg   1/1       Running   0          16m       10.233.92.126   k8s-node1\r\nvolume-configmap                    1/1       Running   5          1d        10.233.76.252   k8s-node3\r\n[root@k8s-node1 svr]# curl 10.233.76.211\r\n<!DOCTYPE html>\r\n<html>\r\n......\r\n</html>\r\n```\r\nPod能根据IP+port正常被访问到，但实际上不建议这样做，\r\n1. Pod不是健壮的，它们可能随时被杀掉重启；\r\n2. 每个Pod都有自己的IP，而且IP是随机变化的；\r\n3. 如果一组Pod有多个相同示例，固定IP的访问起不到负载均衡的作用。\r\n\r\nk8s提出了Service来解决上述问题，我们将从以下几个方面开展此次的实践。\r\n1. Service的定义\r\n2. 内部访问Service\r\n3. 外部访问Service\r\n\r\n# 1、Service的定义\r\n`Service`定义了这样一种抽象：一个`Pod` 的逻辑分组，一种可以访问Pod的策略。 这一组`Pod`能够被`Service`访问到，通常是以下途径实现，\r\n1. 通过Label Selector实现\r\n2. 手动绑定实现\r\n\r\n## **通过Label Selector选择后端Pod**\r\n这里我们来定义Service对象`nginx-service`，通过标签选择器选择后端带有标签`app=nginx`的Pod\r\n```\r\n[root@k8s-node1 svr]# cat << EOF > nginx-service.yaml\r\napiVersion: v1\r\nkind: Service\r\nmetadata:\r\n  name: nginx-service                # Service名称 \r\nspec:\r\n  selector:\r\n    app: nginx                       # 匹配对应Pod的Label\r\n  ports:\r\n  - port: 80                         # Pod端口\r\n    targetPort: 80                   # Service端口\r\n    protocol: TCP                    # 协议 TCP、UDP\r\nEOF\r\n```\r\n执行`kubectl apply`创建并分析Service`nginx-service`。\r\n```\r\n[root@k8s-node1 svr]# kubectl apply -f nginx-service.yaml \r\nservice \"nginx-service\" created\r\n[root@k8s-node1 svr]# kubectl get svc -o wide\r\nNAME            TYPE(1)        CLUSTER-IP(2)    EXTERNAL-IP   PORT(S)   AGE       SELECTOR\r\nkubernetes      ClusterIP   10.233.0.1    <none>        443/TCP   47d       <none>\r\nnginx-service   ClusterIP   10.233.40.0   <none>        80/TCP    3m        app=nginx\r\n```\r\n(1)、通常Service的都会有一个类型，这里默认是ClusterIP，如下所示。\r\n(2)、Service被分配到一个k8s集群内的IP`10.233.40.0`，我们接下来就可以使用这个IP访问后端的Pod了。\r\n```\r\n[root@k8s-node1 svr]# curl 10.233.40.0\r\n<!DOCTYPE html>\r\n<html>\r\n......\r\n</html>\r\n```\r\n通过`kubectl describe`可以查看`nginx-service`与`Pod`的对应关系。\r\n```\r\n[root@k8s-node1 svr]# kubectl describe svc nginx-service \r\nName:              nginx-service\r\nNamespace:         default\r\nLabels:            <none>\r\nAnnotations:       kubectl.kubernetes.io/last-applied-configuration={\"apiVersion\":\"v1\",\"kind\":\"Service\",\"metadata\":{\"annotations\":{},\"name\":\"nginx-service\",\"namespace\":\"default\"},\"spec\":{\"ports\":[{\"port\":80,\"protocol\":\"...\r\nSelector:          app=nginx\r\nType:              ClusterIP\r\nIP:                10.233.40.0\r\nPort:              <unset>  80/TCP\r\nTargetPort:        80/TCP\r\nEndpoints:         10.233.67.31:80,10.233.76.211:80,10.233.92.126:80\r\nSession Affinity:  None\r\nEvents:            <none>\r\n```\r\n我们发现，`nginx-service`与`Pod`通过`Endpoints`(ep)进行了关联，罗列了三个Pod的`IP`和端口。当Service通过Label选择Pod时，Endpoints由Service一同创建，其生命周期与Service相同，Service被删除时，ep会被同步删除。\r\n\r\n## **手动映射服务**\r\n诚如上述所言，`Service`是一组Pod的抽象，但实际上也可以是其他服务的抽象，比如：\r\n- k8s集群外部的mysql、redis等集群\r\n- 将Service指向不同的`Namespace`或者集群\r\n- 迁移部分服务至k8s集群，部分服务依旧不变\r\n\r\n这里我们假设想访问外部的mysql数据库，来看看是怎么使用没有标签的选择器的Service\r\n```bash\r\n[root@k8s-node1 svr]# cat << EOF > mysql-service.yaml\r\napiVersion: v1\r\nkind: Service\r\nmetadata:\r\n  name: mysql-dev\r\nspec:\r\n  ports:\r\n  - port: 3306\r\n    protocol: TCP\r\n    targetPort: 3306\r\n---\r\napiVersion: v1\r\nkind: Endpoints\r\nmetadata:\r\n  name: mysql-dev\r\nsubsets:\r\n- addresses:\r\n  - ip: \"192.168.64.168\"  # 这里假设是外部mysql的地址\r\n  ports:\r\n  - port: 3306\r\nEOF\r\n```\r\n> 需要注意的是，ep的地址不能使环路(127.0.0.1/8)、link-local(192.254.0.0/16)或者link-local多播(224.0.0.0/24)等地址\r\n\r\n使用`kubectl apply`创建Service`mysql-dev`并查看\r\n```\r\n[root@k8s-node1 svr]# kubectl apply -f mysql-service.yaml \r\nservice \"mysql-dev\" created\r\nendpoints \"mysql-dev\" created\r\n[root@k8s-node1 svr]# kubectl get svc -o wide\r\nNAME            TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE       SELECTOR\r\nkubernetes      ClusterIP   10.233.0.1      <none>        443/TCP    51d       <none>\r\nmysql-dev       ClusterIP   10.233.60.190   <none>        3306/TCP   1h        <none>\r\nnginx-service   ClusterIP   10.233.40.0     <none>        80/TCP     4d        app=nginx\r\n```\r\n我们发现Service也已将外部mysql抽象成k8s集群的一个Service对象，其Cluster-IP是`10.233.60.190`，我们来看看能否通过Service访问k8s集群外的mysql服务。\r\n```\r\n[root@k8s-node1 ~]# mysql -uroot -h10.233.60.190 -p\r\nEnter password: \r\nWelcome to the MySQL monitor.  Commands end with ; or \\g.\r\nYour MySQL connection id is 5056\r\nServer version: 5.6.38-log MySQL Community Server (GPL)\r\n\r\nCopyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.\r\n\r\nOracle is a registered trademark of Oracle Corporation and/or its\r\naffiliates. Other names may be trademarks of their respective\r\nowners.\r\n\r\nType 'help;' or '\\h' for help. Type '\\c' to clear the current input statement.\r\n\r\nmysql> show databases;\r\n+--------------------+\r\n| Database           |\r\n+--------------------+\r\n| information_schema |\r\n| cmds               |\r\n| mysql              |\r\n| orange             |\r\n| passport_v1        |\r\n+--------------------+\r\n5 rows in set (0.01 sec)\r\n```\r\n不出意外，可以通过Service的Cluster-IP正常地访问到集群外的mysql服务，那么Cluster-IP究竟是怎么实现路由映射？\r\n\r\n# 2、内部访问Service\r\n## CLUSTER_IP\r\nService的Cluster-IP是一个虚拟IP，由集群Node上的`iptables`规则管理的，`kube-proxy`会监视 Kubernetes master 对 Service 对象和 Endpoints 对象的添加和移除。 \r\n- 对每个 Service，会安装 iptables 规则，从而捕获到达该 Service 的 clusterIP（虚拟 IP）和端口的请求，进而将请求重定向到 Service 的一组 backend 中的某个上面。 \r\n- 对于每个ep对象，也会安装 iptables 规则，这个规则会选择一个 backend Pod。\r\n\r\n> 针对同一服务，不同Node上的iptable规则都是一样的。\r\n\r\n我们通过`iptable-save`命令打印当前节点上`mysql-dev`Service的iptable规则。\r\n```bash\r\n[root@k8s-node1 svr]# iptables-save | grep mysql\r\n-A KUBE-SEP-GLZB56QOTOJL44AT -s 192.168.64.168/32 -m comment --comment \"default/mysql-dev:\" -j KUBE-MARK-MASQ # 4\r\n-A KUBE-SEP-GLZB56QOTOJL44AT -p tcp -m comment --comment \"default/mysql-dev:\" -m tcp -j DNAT --to-destination 192.168.64.168:3306\r\n-A KUBE-SERVICES ! -s 10.233.64.0/18 -d 10.233.60.190/32 -p tcp -m comment --comment \"default/mysql-dev: cluster IP\" -m tcp --dport 3306 -j KUBE-MARK-MASQ # 1\r\n-A KUBE-SERVICES -d 10.233.60.190/32 -p tcp -m comment --comment \"default/mysql-dev: cluster IP\" -m tcp --dport 3306 -j KUBE-SVC-CDJ7VW35W2SPV7CR # 2\r\n-A KUBE-SVC-CDJ7VW35W2SPV7CR -m comment --comment \"default/mysql-dev:\" -j KUBE-SEP-GLZB56QOTOJL44AT # 3\r\n```\r\n这几条iptable规则的含义是：\r\n1、如果集群内的Pod（原地址网段是10.244.0.0/16）要访问`mysql-dev`服务，则允许。\r\n2、其他地址访问，则跳转至规则`KUBE-SVC-CDJ7VW35W2SPV7CR`\r\n3、规则`KUBE-SVC-CDJ7VW35W2SPV7CR`则表示跳转到规则`KUBE-SEP-GLZB56QOTOJL44AT`\r\n4、规则`KUBE-SEP-GLZB56QOTOJL44AT`则将流量跳转至源mysql地址\r\n\r\n> 通常地，会有一组Pod，这时需要通过`spec.sessionAffinity`来定义策略访问这组Pod。\r\n\r\n尽管我们上面可以通过Cluster-IP访问相关Service，但实际使用情况上，我们要优于使用DNS解析，而不是固定IP，那么k8s集群是否支持dns访问呢？\r\n答案是肯定的，除了可以通过Cluster-IP访问Service，k8s还提供了更为方便的`DNS`访问。\r\n\r\n## DNS\r\n前面我们查看`Deployment时`，会发现其中有`kube-dns`，原来k8s在部署时，会默认安装kube-dns组件。\r\n```\r\n[root@k8s-node1 svr]# kubectl get deployment -n kube-system \r\nNAME                   DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE\r\nheapster               1         1         1            1           51d\r\nkube-dns               2         2         2            2           51d\r\n......\r\n```\r\n`kube-dns`是一个DNS服务器，负责管理k8s集群的映射记录。每当新Service被创建，`kube-dns`会添加该 Service的DNS记录。集群中的Pod容器可以通过`<SERVICE_NAME>.<NAMESPACE_NAME>`访问对应Service。\r\n```\r\n[root@k8s-node1 svr]# kubectl run busybox --rm -ti --image=busybox /bin/sh\r\nIf you don't see a command prompt, try pressing enter.\r\n/ # nslookup nginx-service.default\r\nServer:    10.233.0.3\r\nAddress 1: 10.233.0.3 kube-dns.kube-system.svc.cluster.local\r\n\r\nName:      nginx-service.default\r\nAddress 1: 10.233.40.0 nginx-service.default.svc.cluster.local\r\n```\r\n通过`nslookup`发现‘域名’`nginx-service.default`被成功解析成Cluster-IP`10.233.40.0`，其真正域名是`nginx-service.default.svc.cluster.local`。\r\n\r\n如果我们要访问其他NameSpace里面的Service，那么要带上Service所在的Namespace，可以通过`kubectl get namespace`查看。\r\n```\r\n[root@k8s-node1 svr]# kubectl get namespace\r\nNAME          STATUS    AGE\r\ndefault       Active    51d\r\nkube-public   Active    51d\r\nkube-system   Active    51d\r\n```\r\n\r\n# 3、外部访问Service\r\n服务都有了，也可以集群内部正常被访问到，但很多情况下，我们都希望Service能够向外暴露，k8s提供了多种类型的Service，默认就是我们上述所说的`Cluster-IP`。\r\n| 类型 | 说明 | 示例 |\r\n| --- | --- | --- |\r\n| ClusterIP | 通过集群内部IP暴露服务，服务只能够在集群内部可以访问，默认值。\r\n| NodePort | 通过每个`Node`的IP和静态端口（NodePort）暴露服务，服务会路由到`ClusterIP`服务，这个 ClusterIP 服务会自动创建。\r\n| LoadBalancer | 使用云提供商的负载均衡器，可以向外部暴露服务。\r\n| ExternalName | 通过返回 CNAME 和它的值，可以将服务映射到 externalName 字段的内容（例如， foo.bar.example.com）。\r\n\r\n这里我们来看看`NodePort`类型的Service与ClusterIP有何不同。\r\n首先，我们修改上面的Service定义`nginx-service`，将其类型设置成NodePort，。\r\n```\r\n[root@k8s-node1 svr]# cat << EOF > nginx-service.yaml\r\napiVersion: v1\r\nkind: Service\r\nmetadata:\r\n  name: nginx-service                # Service名称 \r\nspec:\r\n  type: NodePort\r\n  selector:\r\n    app: nginx                       # 匹配对应Pod的Label\r\n  ports:\r\n  - port: 80                         # Pod端口\r\n    targetPort: 80                   # Service端口\r\n    protocol: TCP                    # 协议 TCP、UDP\r\nEOF\r\n```\r\n然后，重新创建Service。\r\n```\r\n[root@k8s-node1 svr]# kubectl apply -f nginx-service.yaml \r\nservice \"nginx-service\" configured\r\n[root@k8s-node1 svr]# kubectl get svc -o wide\r\nNAME            TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE       SELECTOR\r\nkubernetes      ClusterIP   10.233.0.1      <none>        443/TCP        51d       <none>\r\nmysql-dev       ClusterIP   10.233.60.190   <none>        3306/TCP       2h        <none>\r\nnginx-service   NodePort    10.233.40.0     <none>        80:30997/TCP   4d        app=nginx\r\n```\r\nKubernetes 依然会为`nginx-service`分配一个 ClusterIP，不同的是：\r\n\r\n1、EXTERNAL-IP为这时为`nodes`，表示可通过 Cluster 每个节点自身的 IP 访问 Service。\r\n\r\n2、PORT(S) 为 8080:32312。8080 是 ClusterIP 监听的端口，32312 则是节点上监听的端口。Kubernetes 会从 30000-32767 中分配一个可用的端口，每个节点都会监听此端口并将请求转发给 Service。\r\n\r\n> NodePort也可以通过`spec.ports.nodePort指定`\r\n\r\n我们通过NodeIP:NodePort来测试服务是否正常对外暴露。\r\n```\r\n[root@k8s-node1 svr]# curl localhost:30997\r\n<!DOCTYPE html>\r\n<html>\r\n.....\r\n</html>\r\n```\r\n> 集群上所有节点都能够访问`nginx-service`\r\n\r\n服务正常对外暴露，那么k8s是如何将<NodeIP>:<NodePort>映射到Pod的呢？你可能也会猜测是通过`iptables`，聪明如你，与`ClusterIP`一样，也是借助了`iptables`，我们来看看路由规则是怎么定义的。\r\n```\r\n[root@k8s-node1 svr]# iptables-save | grep nginx\r\n-A KUBE-NODEPORTS -p tcp -m comment --comment \"default/nginx-service:\" -m tcp --dport 30997 -j KUBE-MARK-MASQ\r\n-A KUBE-NODEPORTS -p tcp -m comment --comment \"default/nginx-service:\" -m tcp --dport 30997 -j KUBE-SVC-GKN7Y2BSGW4NJTYL # 1\r\n-A KUBE-SEP-IHCVQUC34BKABT4U -s 10.233.92.126/32 -m comment --comment \"default/nginx-service:\" -j KUBE-MARK-MASQ\r\n-A KUBE-SEP-IHCVQUC34BKABT4U -p tcp -m comment --comment \"default/nginx-service:\" -m tcp -j DNAT --to-destination 10.233.92.126:80\r\n-A KUBE-SEP-JJZ45HCOVL5AGIB5 -s 10.233.76.211/32 -m comment --comment \"default/nginx-service:\" -j KUBE-MARK-MASQ\r\n-A KUBE-SEP-JJZ45HCOVL5AGIB5 -p tcp -m comment --comment \"default/nginx-service:\" -m tcp -j DNAT --to-destination 10.233.76.211:80\r\n-A KUBE-SEP-QGOYFQDKT6EC4BBN -s 10.233.67.31/32 -m comment --comment \"default/nginx-service:\" -j KUBE-MARK-MASQ\r\n-A KUBE-SEP-QGOYFQDKT6EC4BBN -p tcp -m comment --comment \"default/nginx-service:\" -m tcp -j DNAT --to-destination 10.233.67.31:80\r\n-A KUBE-SERVICES ! -s 10.233.64.0/18 -d 10.233.40.0/32 -p tcp -m comment --comment \"default/nginx-service: cluster IP\" -m tcp --dport 80 -j KUBE-MARK-MASQ\r\n-A KUBE-SERVICES -d 10.233.40.0/32 -p tcp -m comment --comment \"default/nginx-service: cluster IP\" -m tcp --dport 80 -j KUBE-SVC-GKN7Y2BSGW4NJTYL\r\n-A KUBE-SVC-GKN7Y2BSGW4NJTYL -m comment --comment \"default/nginx-service:\" -m statistic --mode random --probability 0.33332999982 -j KUBE-SEP-QGOYFQDKT6EC4BBN\r\n-A KUBE-SVC-GKN7Y2BSGW4NJTYL -m comment --comment \"default/nginx-service:\" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-JJZ45HCOVL5AGIB5\r\n-A KUBE-SVC-GKN7Y2BSGW4NJTYL -m comment --comment \"default/nginx-service:\" -j KUBE-SEP-IHCVQUC34BKABT4U\r\n```\r\n关于端口的路由规则解析如下：\r\n(1)、访问当前节点30997端口的请求会应用路由规则`KUBE-SVC-GKN7Y2BSGW4NJTYL`\r\n(2)、路由规则`KUBE-SVC-GKN7Y2BSGW4NJTYL`就是随机负载到每一个Pod，这里是三个Pod\r\n\r\n关于Service的应用实践就到这里，接下来我们将展开Job的学习实践。\r\n\r\n# 参考文档\r\n[Service](https://kubernetes.io/docs/concepts/services-networking/service/)"}}
{"actor":{"display_login":"kaybinwong"}}