<p>I have a working insecure K8S cluster setup: CoreOS alpha image + Vagrant (custom solution follow K8S getting start guide of scratch setup). Now I want to setup the authentication for K8s Cluster Admins who can access API via <code>kubectl cluster-info</code> command etc. I want to setup something similar like <a href="https://github.com/kubernetes/kubernetes/blob/master/docs/design/access.md#design" rel="nofollow">design doc</a> - Simple profile.</p>

<p>Then I followed <a href="https://github.com/kubernetes/kubernetes/blob/968cbbee5d4964bd916ba379904c469abb53d623/docs/admin/authentication.md" rel="nofollow">authentication</a> docs, I picked the <strong>Client certificate authentication</strong> for authentication plugins. </p>

<p>I prepared <a href="http://kubernetes.io/v1.0/docs/getting-started-guides/scratch.html#preparing-certs" rel="nofollow">certs</a>, saved <code>/srv/kubernetes/ca.crt</code>,
<code>/srv/kubernetes/server.crt</code>, <code>/srv/kubernetes/server.key</code> on the <strong>Master Node</strong>.</p>

<p>I also setup the <code>kubeconfig</code> file by following the guide.</p>

<pre><code>kubectl config set-cluster $CLUSTER_NAME --certificate-authority=$CA_CERT --embed-certs=true --server=https://$MASTER_IP
kubectl config set-credentials $CLUSTER_NAME --client-certificate=$CLI_CERT --client-key=$CLI_KEY --embed-certs=true --token=$TOKEN
kubectl config set-context $CLUSTER_NAME --cluster=$CLUSTER_NAME --user=admin
kubectl config use-context $CONTEXT --cluster=$CONTEXT
</code></pre>

<p>When api-server starts, it also use the same value. see <code>$CA_CERT</code>, <code>$CLI_CERT</code>, <code>$CLI_KEY</code>. Q1: are those vlaues in the right place?</p>

<pre><code>/kube-apiserver \
--allow_privileged=true \
--bind_address=0.0.0.0 \
--secure_port=6443 \
--kubelet_https=true \
--service-cluster-ip-range=${SERVICE_CLUSTER_IP_RANGE} \
--etcd_servers=$ETCD_SERVER \
--service-node-port-range=${SERVICE_NODE_PORT_RANGE} \
--cluster-name=$CLUSTER_NAME \
--client-ca-file=$CA_CERT \
--tls-cert-file=$CLI_CERT \
--tls-private-key-file=$CLI_KEY \
--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota \
--logtostderr=true 
</code></pre>

<p>Logs are below</p>

<pre><code>Aug 30 06:31:30 kube-master docker[3706]: E0830 06:31:30.373083       1 reflector.go:136] Failed to list *api.ResourceQuota: Get http://127.0.0.1:8080/api/v1/resourcequotas: dial tcp 127.0.0.1:8080: connection refused
Aug 30 06:31:30 kube-master docker[3706]: E0830 06:31:30.373523       1 reflector.go:136] Failed to list *api.Secret: Get http://127.0.0.1:8080/api/v1/secrets?fieldSelector=type%3Dkubernetes.io%2Fservice-account-token: dial tcp 127.0.0.1:8080: connection refused
Aug 30 06:31:30 kube-master docker[3706]: E0830 06:31:30.373631       1 reflector.go:136] Failed to list *api.ServiceAccount: Get http://127.0.0.1:8080/api/v1/serviceaccounts: dial tcp 127.0.0.1:8080: connection refused
Aug 30 06:31:30 kube-master docker[3706]: E0830 06:31:30.373695       1 reflector.go:136] Failed to list *api.LimitRange: Get http://127.0.0.1:8080/api/v1/limitranges: dial tcp 127.0.0.1:8080: connection refused
Aug 30 06:31:30 kube-master docker[3706]: E0830 06:31:30.373748       1 reflector.go:136] Failed to list *api.Namespace: Get http://127.0.0.1:8080/api/v1/namespaces: dial tcp 127.0.0.1:8080: connection refused
Aug 30 06:31:30 kube-master docker[3706]: E0830 06:31:30.373788       1 reflector.go:136] Failed to list *api.Namespace: Get http://127.0.0.1:8080/api/v1/namespaces: dial tcp 127.0.0.1:8080: connection refused
Aug 30 06:31:30 kube-master docker[3706]: [restful] 2015/08/30 06:31:30 log.go:30: [restful/swagger] listing is available at https://10.0.2.15:6443/swaggerapi/
Aug 30 06:31:30 kube-master docker[3706]: [restful] 2015/08/30 06:31:30 log.go:30: [restful/swagger] https://10.0.2.15:6443/swaggerui/ is mapped to folder /swagger-ui/
Aug 30 06:31:30 kube-master docker[3706]: I0830 06:31:30.398612       1 server.go:441] Serving securely on 0.0.0.0:6443
Aug 30 06:31:30 kube-master docker[3706]: I0830 06:31:30.399042       1 server.go:483] Serving insecurely on 127.0.0.1:8080
</code></pre>

<p>On my <strong>MacOS</strong> machine, I want to connect <code>kubectl</code> to my <strong>$CLUSTER_NAME</strong> cluster.</p>

<pre><code>export KUBERNETES_MASTER=http://172.17.8.100:6443

kubectl cluster-info
</code></pre>

<p>Terminal outputs:</p>

<pre><code> ➜ kubectl cluster-info                                                                                                                                                                              
error: couldn't read version from server: Get http://172.17.8.100:6443/api: malformed HTTP response "\x15\x03\x01\x00\x02\x02"
</code></pre>

<p>Here is my <code>kubeconfig</code> file on <em>MacOS machine</em> <code>~/.kube/config</code></p>

<pre><code> ➜ kubectl config view                                                                                                                                                                             
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: REDACTED
    server: http://172.17.8.100:6443
  name: kube-01
contexts:
- context:
    cluster: kube-01
    user: admin
  name: kube
current-context: kube
kind: Config
preferences: {}
users:
- name: admin
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED
    token: cxKranwtWI2nyASebbF1HV3p1EWJbNcE
</code></pre>

<p><strong>Q: How could my <code>kubectl</code> on MacOS to access my K8S cluster securely? since I never add user <code>admin</code> on my api-server, I assume that all authentication is being done by <code>ca-file</code>?</strong></p>

<p><strong>Q: Once I fix the secure login issue, how could I update the <code>admission-control</code> plugins api error issue like <code>ServiceAccount</code> connection refuse above?</strong></p>

<p><strong>Q: Do I use <code>http</code> or <code>https</code>? I prefer to use <code>http://IP:6443</code>, not sure it is the problem?</strong></p>

<p><strong>Q: Do I need to apply <code>--token-auth-file=</code> or <code>--basic-auth-file</code>? By reading the Docs, I think I could pick one of the method for authentication. I would prefer to do it in <code>ca</code> which is more secure, right?</strong></p>

<hr>

<p>I used <code>see function create-certs in cluster/gce/util.sh</code> to generate my <code>certs</code> files. I am not too familiar to <code>certs</code> and <code>keys</code>, so that I post them here. Well, it is really a dummy <code>certs</code> and <code>keys</code> for testing. It is not being used anywhere. Simply posted here to varify if I did something wrong here. </p>

<p><strong>ca.crt</strong></p>

<p>-----BEGIN CERTIFICATE-----
    MIIDWTCCAkGgAwIBAgIJAMbTBaUcQSbGMA0GCSqGSIb3DQEBCwUAMCIxIDAeBgNV
    BAMMFzE3Mi4xNy44LjEwMEAxNDQwNzgwMjgxMB4XDTE1MDgyODE2NDQ0MVoXDTI1
    MDgyNTE2NDQ0MVowIjEgMB4GA1UEAwwXMTcyLjE3LjguMTAwQDE0NDA3ODAyODEw
    ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDNmT0O8sBXTd2Htbb+hnsq
    P/YvUNYTXzLy6+T/d9/KRrxq1JWO70E7L2hFOvOdGF0gZuoAefki5ymkFYfwoZsK
    NEXvA1AxBMtQnMCdUOp7m5XW+c9uFepW+jzvb4PRBoUHZjW5HhxT6UZ21FiEvwHP
    NBnCL9gp1NIcNOaUIZvFI7hpko0tfAPFYY0NkHRo6mLpvzaGTippzySMSLyQ7cs4
    IcUrFGJbsTNISCSsCG//+A6I62sQAURr0hjeW9FmGHxwYW+0wdyyTtlFPTKrVrC4
    ETc5WeQoJeZhjoH7Dkj8l6QBvv2cDtZwnY2oCUGXf63c3hoRaEkeFis1RWQcQKoT
    AgMBAAGjgZEwgY4wHQYDVR0OBBYEFONIYbWt3l9D5j9VvJADUQfmIBpQMFIGA1Ud
    IwRLMEmAFONIYbWt3l9D5j9VvJADUQfmIBpQoSakJDAiMSAwHgYDVQQDDBcxNzIu
    MTcuOC4xMDBAMTQ0MDc4MDI4MYIJAMbTBaUcQSbGMAwGA1UdEwQFMAMBAf8wCwYD
    VR0PBAQDAgEGMA0GCSqGSIb3DQEBCwUAA4IBAQCJtrf1Mf+pHwCsMG8HPcuR4oij
    ugYkzawEF2FSCe2VbFMDxwmHbHw2N9ZOwRLyeSuR0JAY5aN31pqIzYCmmKf2otKU
    +mtTaK5YIsZU2IdxoR6VHaHT83zSGq9RhteqDdM8tuMvNsV5I9pJCu+Bkv3MsJpN
    0PIc+GFs52A+bQC3cjWqLkgJeYEqolNnJpeex9G3ovqbTzavgM8q5gjdTyz8tDIo
    Dc4RKcuwyrAnkiJ93HdWLwkKcEXzrX/lU9NYsvmycBVbkRaIh7md82HCUiwkmmJC
    Xz3+xVrghzMo0DgoInzxcPFRWPc00CZcb5P5VRepa2rPwEyNgEp3BsQLXFIt
    -----END CERTIFICATE-----</p>

<p><strong>server.crt</strong></p>

<pre><code>Certificate:
        Data:
            Version: 3 (0x2)
            Serial Number: 1 (0x1)
            Signature Algorithm: sha256WithRSAEncryption
            Issuer: CN=172.17.8.100@1440780281
            Validity
                Not Before: Aug 28 16:44:41 2015 GMT
                Not After : Aug 25 16:44:41 2025 GMT
            Subject: CN=kube-master
            Subject Public Key Info:
                Public Key Algorithm: rsaEncryption
                RSA Public Key: (2048 bit)
                    Modulus (2048 bit):
                        00:ab:3f:cf:95:50:3d:7f:b4:82:ba:72:7a:88:2e:
                        41:79:67:7d:9a:4a:22:27:5f:fd:5c:78:6f:3d:ad:
                        57:4c:fd:37:9e:b5:35:f1:88:59:c1:e9:10:38:3e:
                        de:7f:57:cf:e9:fc:fd:d7:b5:a8:7a:0e:5f:e4:16:
                        6f:2a:66:98:28:6c:42:a8:5f:95:3d:0b:02:f2:ec:
                        ab:aa:19:40:60:b3:e5:7a:64:7d:5b:f2:9c:84:d5:
                        bb:06:79:e7:00:2f:2c:a0:0a:88:f4:b0:c5:31:de:
                        7d:30:d6:b3:4d:ea:64:85:bb:f9:89:5a:f5:22:41:
                        92:35:d4:a4:7d:80:64:65:d9:1d:c9:30:39:af:34:
                        57:cd:d5:56:5d:9f:35:5d:ee:a3:07:ed:f1:c5:68:
                        db:db:12:65:31:e6:6c:1e:77:44:3e:7c:03:bc:89:
                        f0:4c:14:a6:41:39:22:a3:a3:a0:8d:20:eb:69:7a:
                        c5:de:b0:2f:94:67:68:ab:8c:8a:24:59:38:a4:57:
                        19:2d:c2:0e:37:c8:73:98:ae:d8:0a:a4:e2:72:22:
                        49:9a:55:58:ad:8e:c3:eb:42:b5:41:02:c9:40:27:
                        d1:77:41:ab:4f:0b:2a:6b:b2:b6:38:7f:a0:ce:cf:
                        9f:cd:7c:54:72:c6:43:cd:1d:5b:60:b9:45:eb:10:
                        ab:ad
                    Exponent: 65537 (0x10001)
            X509v3 extensions:
                X509v3 Basic Constraints: 
                    CA:FALSE
                X509v3 Subject Key Identifier: 
                    B2:46:5F:5A:68:3E:08:78:25:8C:AE:5E:EB:F1:3B:7B:CF:9D:A6:F3
                X509v3 Authority Key Identifier: 
                    keyid:E3:48:61:B5:AD:DE:5F:43:E6:3F:55:BC:90:03:51:07:E6:20:1A:50
                    DirName:/CN=172.17.8.100@1440780281
                    serial:C6:D3:05:A5:1C:41:26:C6

                X509v3 Extended Key Usage: 
                    TLS Web Server Authentication
                X509v3 Key Usage: 
                    Digital Signature, Key Encipherment
                X509v3 Subject Alternative Name: 
                    IP Address:172.17.8.100, IP Address:10.100.0.1, DNS:kubernetes, DNS:kubernetes.default, DNS:kubernetes.default.svc, DNS:kubernetes.default.svc.cluster.local, DNS:kube-master
        Signature Algorithm: sha256WithRSAEncryption
            58:b1:63:41:3e:94:ed:3d:bd:3c:e8:0c:78:30:54:c1:6d:33:
            00:42:74:c8:7a:64:cc:fd:9a:70:ab:38:5b:1c:92:7c:9b:56:
            1a:d7:fd:38:51:07:cf:5a:b5:0a:11:85:01:3d:52:86:96:ad:
            16:be:ea:9c:2c:ee:3c:14:c9:5b:58:d7:ab:45:ae:d8:e0:2d:
            70:7c:55:40:44:b8:98:ad:1b:d4:66:35:c5:78:13:4c:e7:5a:
            de:82:15:43:cb:bb:83:3a:09:04:fa:5e:6f:d9:ca:17:b8:40:
            00:b0:ba:06:ed:73:ed:c8:c7:5a:53:aa:d3:43:a2:f1:c2:cf:
            14:9b:c2:7b:b7:c0:2a:56:a0:53:2e:af:2d:07:65:c0:70:c1:
            92:86:34:05:39:3c:ed:3f:6e:f9:31:7f:de:5a:ed:9b:c8:83:
            e0:f4:9c:de:c7:9c:04:be:d2:6e:8d:5e:3e:ad:46:d4:82:70:
            9d:79:b9:c3:dd:b4:c0:6e:1b:23:d0:45:be:26:c6:7e:4c:ec:
            c5:c3:c9:ee:1e:93:d4:a5:11:e9:6a:1d:e1:ee:af:eb:83:e6:
            dd:ec:13:7b:45:60:18:f5:05:3f:61:7b:3c:2b:b1:28:c4:92:
            5e:bc:67:c0:02:22:a9:aa:69:d5:e9:0e:75:80:36:b2:66:84:
            fe:05:c2:75
    -----BEGIN CERTIFICATE-----
    MIID3DCCAsSgAwIBAgIBATANBgkqhkiG9w0BAQsFADAiMSAwHgYDVQQDDBcxNzIu
    MTcuOC4xMDBAMTQ0MDc4MDI4MTAeFw0xNTA4MjgxNjQ0NDFaFw0yNTA4MjUxNjQ0
    NDFaMBYxFDASBgNVBAMMC2t1YmUtbWFzdGVyMIIBIjANBgkqhkiG9w0BAQEFAAOC
    AQ8AMIIBCgKCAQEAqz/PlVA9f7SCunJ6iC5BeWd9mkoiJ1/9XHhvPa1XTP03nrU1
    8YhZwekQOD7ef1fP6fz917Woeg5f5BZvKmaYKGxCqF+VPQsC8uyrqhlAYLPlemR9
    W/KchNW7BnnnAC8soAqI9LDFMd59MNazTepkhbv5iVr1IkGSNdSkfYBkZdkdyTA5
    rzRXzdVWXZ81Xe6jB+3xxWjb2xJlMeZsHndEPnwDvInwTBSmQTkio6OgjSDraXrF
    3rAvlGdoq4yKJFk4pFcZLcION8hzmK7YCqTiciJJmlVYrY7D60K1QQLJQCfRd0Gr
    Twsqa7K2OH+gzs+fzXxUcsZDzR1bYLlF6xCrrQIDAQABo4IBJzCCASMwCQYDVR0T
    BAIwADAdBgNVHQ4EFgQUskZfWmg+CHgljK5e6/E7e8+dpvMwUgYDVR0jBEswSYAU
    40hhta3eX0PmP1W8kANRB+YgGlChJqQkMCIxIDAeBgNVBAMMFzE3Mi4xNy44LjEw
    MEAxNDQwNzgwMjgxggkAxtMFpRxBJsYwEwYDVR0lBAwwCgYIKwYBBQUHAwEwCwYD
    VR0PBAQDAgWgMIGABgNVHREEeTB3hwSsEQhkhwQKZAABggprdWJlcm5ldGVzghJr
    dWJlcm5ldGVzLmRlZmF1bHSCFmt1YmVybmV0ZXMuZGVmYXVsdC5zdmOCJGt1YmVy
    bmV0ZXMuZGVmYXVsdC5zdmMuY2x1c3Rlci5sb2NhbIILa3ViZS1tYXN0ZXIwDQYJ
    KoZIhvcNAQELBQADggEBAFixY0E+lO09vTzoDHgwVMFtMwBCdMh6ZMz9mnCrOFsc
    knybVhrX/ThRB89atQoRhQE9UoaWrRa+6pws7jwUyVtY16tFrtjgLXB8VUBEuJit
    G9RmNcV4E0znWt6CFUPLu4M6CQT6Xm/Zyhe4QACwugbtc+3Ix1pTqtNDovHCzxSb
    wnu3wCpWoFMury0HZcBwwZKGNAU5PO0/bvkxf95a7ZvIg+D0nN7HnAS+0m6NXj6t
    RtSCcJ15ucPdtMBuGyPQRb4mxn5M7MXDye4ek9SlEelqHeHur+uD5t3sE3tFYBj1
    BT9hezwrsSjEkl68Z8ACIqmqadXpDnWANrJmhP4FwnU=
    -----END CERTIFICATE-----
</code></pre>

<p><strong>server.key</strong></p>

<p>-----BEGIN RSA PRIVATE KEY-----
    MIIEpAIBAAKCAQEAqz/PlVA9f7SCunJ6iC5BeWd9mkoiJ1/9XHhvPa1XTP03nrU1
    8YhZwekQOD7ef1fP6fz917Woeg5f5BZvKmaYKGxCqF+VPQsC8uyrqhlAYLPlemR9
    W/KchNW7BnnnAC8soAqI9LDFMd59MNazTepkhbv5iVr1IkGSNdSkfYBkZdkdyTA5
    rzRXzdVWXZ81Xe6jB+3xxWjb2xJlMeZsHndEPnwDvInwTBSmQTkio6OgjSDraXrF
    3rAvlGdoq4yKJFk4pFcZLcION8hzmK7YCqTiciJJmlVYrY7D60K1QQLJQCfRd0Gr
    Twsqa7K2OH+gzs+fzXxUcsZDzR1bYLlF6xCrrQIDAQABAoIBAAtfMWm46lyQoB3B
    fGGOsMpfFPgp9BqpRSne1YRC/okeR5NCdVKUu2ElGO6jPiM2sZfYNQMeDRIN4lBD
    LR6jsXb9uW906XQkRw3aqYuiIaRKTfLSuYBhnAM2LjU/4xcgCtaV3IJjOrUVETst
    Brsl1YcL9IYqhBzCPfNVK5cp74DTzleBjl7ng1y8ijGOTcp5JwUbrrQQZ0U9uqjS
    nCAjB63e8x7JswXx1jo4pDeumJzyJ1eHNA0oXwSbgZ/q/oUHHYykUrFkPYIIAMKu
    lZO/Lh2tRNdDf8lXupWmhfcwDO9DYcRr4v37hnDqknWWHEdgR9hborc6vZYAMpPB
    0LrIfAECgYEA0rT7bFDCCBmk5yDw2cOl1CHT1BTq7Elw2cjAGgjAygx0puGKuBnr
    qBYeAQqx3ZZHlMsiT3gSbRP9CLws+QgSUf87deM0kBoiWG6m+KgSxmBIMRJCdo+S
    c+3QZwWLBFHQLaJCDRN4XNr1HuHzcKYO4th/SpDZ3lQc9wO7S3dBHpsCgYEA0A+B
    ogw30zf1rIaIv8rRMOItqA6pgR6DbspAYexZyEKUexsvHOw6KMDRz7IwzZRVUkjI
    uPfEkq3qAhYpEgzi/BIsnj/Ku91THkzkkDBolpuJAa068GupQgbLCLhKWa1h7qrI
    mAFOxy+9ZIFWbmy4UDaqgT5O78gw1CFwibYXn1cCgYEAlDPX5AepcikXY7o3rfN+
    4AYrCDDuS+QcDBK3i5g8geDg68AX4gXZSxDDadgr4r+g+XcnWt4Jl89HWq2AtGiI
    +kObfv+gKPs4zRqHNr6A9icin+FH/jxdtky/GLc9YHxrAK3v52KadjVL07z5jXI/
    Zi8A2WGo3EgtV1C4nAv1MaECgYAp0GP6IEB754wtLyB+gxFFpL8OPlwcgfhiJK2J
    wIlOsOrMTutKAcOyewXvmt0qA7yd+9izK8BKxj74SmHYqdRYWoKzDxj8Zn+U4Fkz
    DTeHxRxkxN7KgKiUh274gqkWmrzKzXHg8qpVZ6fFciTfrmPgYwwjS1Vr5SzDBTFr
    y7e1owKBgQDMKHPuEE9LT3ljiZFIoU6yxbWU/+rMaJwqmV5bEXbfrL06PjTw7kp/
    UnLHJ3TVdCXnY2J4Si39cYAhL5Wr5JiubviaW5zCjjOXbrE3ck16kkJsS8DOXjHT
    nHNGV48GE51THWl/NbuRQz/rD9McsCwixNm66C2EiakKuKLuv3tI3Q==
    -----END RSA PRIVATE KEY-----</p>

<p>I think you may have the exact same issue that I just solved. I believe it was you who asked a similar question on #google-containers and something that user "vishh" said fixed the problem for me. Make sure your master IP/hostname is in the certificate that you use for your api server under the cert's <code>Subject Alternative Name:</code> section.</p>

<pre><code>$ openssl x509 -in kube-apiserver-server.pem -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            1d:60:b0:98:70:95:23:f8
        Signature Algorithm: sha256WithRSAEncryption
        ...
        ...
        ...

            X509v3 Subject Alternative Name:
                DNS:*.kubestack.io, DNS:*.c.kubestack.internal, IP Address:127.0.0.1, IP Address:192.168.10.50
</code></pre>

<p>I have very little experience with certs and keys so I used the guide <a href="https://github.com/kelseyhightower/docker-kubernetes-tls-guide" rel="nofollow">here</a> to generate mine.</p>

<p>You shouldn't need to use any other auth flags (token/basic), it's done through the certs as you assumed.</p>

<p>You need to use http<strong>s</strong> when specifying the server.</p>

<p>I'm unsure about your admission-control question.</p>

<p>Hope this helps.</p>
