{"action":"created","issue":{"url":"https://api.github.com/repos/spinnaker/spinnaker/issues/2900","repository_url":"https://api.github.com/repos/spinnaker/spinnaker","labels_url":"https://api.github.com/repos/spinnaker/spinnaker/issues/2900/labels{/name}","comments_url":"https://api.github.com/repos/spinnaker/spinnaker/issues/2900/comments","events_url":"https://api.github.com/repos/spinnaker/spinnaker/issues/2900/events","html_url":"https://github.com/spinnaker/spinnaker/issues/2900","id":331817738,"node_id":"MDU6SXNzdWUzMzE4MTc3Mzg=","number":2900,"title":"spin-clouddriver gcloud command not found","user":{"login":"HirokiSakonju","id":4677344,"node_id":"MDQ6VXNlcjQ2NzczNDQ=","avatar_url":"https://avatars1.githubusercontent.com/u/4677344?v=4","gravatar_id":"","url":"https://api.github.com/users/HirokiSakonju","html_url":"https://github.com/HirokiSakonju","followers_url":"https://api.github.com/users/HirokiSakonju/followers","following_url":"https://api.github.com/users/HirokiSakonju/following{/other_user}","gists_url":"https://api.github.com/users/HirokiSakonju/gists{/gist_id}","starred_url":"https://api.github.com/users/HirokiSakonju/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HirokiSakonju/subscriptions","organizations_url":"https://api.github.com/users/HirokiSakonju/orgs","repos_url":"https://api.github.com/users/HirokiSakonju/repos","events_url":"https://api.github.com/users/HirokiSakonju/events{/privacy}","received_events_url":"https://api.github.com/users/HirokiSakonju/received_events","type":"User","site_admin":false},"labels":[{"id":916190248,"node_id":"MDU6TGFiZWw5MTYxOTAyNDg=","url":"https://api.github.com/repos/spinnaker/spinnaker/labels/provider/kubernetes-v2","name":"provider/kubernetes-v2","color":"207de5","default":false},{"id":111166992,"node_id":"MDU6TGFiZWwxMTExNjY5OTI=","url":"https://api.github.com/repos/spinnaker/spinnaker/labels/question","name":"question","color":"cc317c","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-06-13T02:21:23Z","updated_at":"2018-06-14T14:42:43Z","closed_at":null,"author_association":"NONE","body":"### Issue Summary:\r\nI deploy spinnaker, but spin-clouddriver is `Readiness probe failed`.  \r\nI checked the log with `kubectl logs pod-name-n spinnaker` and an internal server error has occurred.  \r\n\r\n### Cloud Provider(s):\r\nKubernetes\r\n\r\n### Environment:\r\nHalyard:  \r\non GCE(ubuntu-1604-xenial-v20180522)\r\n```\r\n$ hal -v\r\n1.2.0-20180509113540\r\n```\r\n\r\nspinnaker:\r\non GKE(1.10.4-gke.0)  \r\n```\r\n$ hal config version edit --version 1.7.6\r\n```\r\n```\r\n$ hal config provider kubernetes account add spinnaker \\\r\n    --provider-version v2 \\\r\n    --context $(kubectl config current-context)\r\n```\r\n\r\n### Feature Area: \r\nclouddriver\r\n\r\n### Description:\r\nThe status of pod is below and the status of spin-clouddriver is not normal.\r\n```\r\n$ kubectl get pod -n spinnaker\r\nNAME                                READY     STATUS    RESTARTS   AGE\r\nspin-clouddriver-8585c8cc88-pnsqb   0/1       Running   0          23m\r\nspin-deck-64c77858bc-j2bjf          1/1       Running   0          23m\r\nspin-echo-5b778876dd-5kfzx          1/1       Running   0          23m\r\nspin-front50-65c677f697-fv5xx       1/1       Running   0          23m\r\nspin-gate-7655445d46-9jlb7          1/1       Running   0          23m\r\nspin-orca-b88b864d6-c479t           1/1       Running   0          22m\r\nspin-redis-6d49c9c5b9-x4vkw         1/1       Running   0          23m\r\nspin-rosco-7d8cc5bb6f-5gwj6         1/1       Running   0          22m\r\n\r\n```\r\n\r\n### Steps to Reproduce:\r\n\r\n1. install halyard on GCE(ubuntu-1604-xenial-v20180522)\r\n2. `hal config provider kubernetes enable`\r\n3. ```hal config provider kubernetes account add spinnaker \\\r\n    --provider-version v2 \\\r\n    --context $(kubectl config current-context)\r\n   ```\r\n4. `hal config features edit --artifacts true`\r\n5. `hal config deploy edit --type distributed --account-name spinnaker`\r\n6. storage edit\r\n   ```hal config storage gcs edit --project $PROJECT \\\r\n    --json-path $SERVICE_ACCOUNT_DEST \\\r\n    --bucket $BUCKET_NAME\r\n    ```\r\n7. `hal config version edit --version 1.7.6`\r\n8. `hal deploy apply`\r\n\r\n\r\n### Additional Details:\r\n\r\n```\r\n2018-06-13 02:03:48.418 ERROR 1 --- [0.0-7002-exec-3] c.n.s.k.w.e.GenericExceptionHandlers     : Internal Server Error\r\n\r\njava.lang.RuntimeException: com.netflix.spinnaker.clouddriver.kubernetes.v2.op.job.KubectlJobExecutor$KubectlException: Failed to read [namespace] from : Unable to connect to the server: error executing access token command \"/usr/lib/google-cloud-sdk/bin/gcloud config config-helper --format=json\": err=fork/exec /usr/lib/google-cloud-sdk/bin/gcloud: no such file or directory output= stderr=\r\n\r\n\tat com.netflix.spinnaker.clouddriver.kubernetes.v2.security.KubernetesV2Credentials.getDeclaredNamespaces(KubernetesV2Credentials.java:311) ~[clouddriver-kubernetes-2.22.0-SNAPSHOT.jar:2.22.0-SNAPSHOT]\r\n\tat com.netflix.spinnaker.clouddriver.kubernetes.security.KubernetesCredentials$getDeclaredNamespaces.call(Unknown Source) ~[na:na]\r\n\tat com.netflix.spinnaker.clouddriver.kubernetes.health.KubernetesHealthIndicator.checkHealth(KubernetesHealthIndicator.groovy:65) ~[clouddriver-kubernetes-2.22.0-SNAPSHOT.jar:2.22.0-SNAPSHOT]\r\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_151]\r\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_151]\r\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_151]\r\n\tat java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_151]\r\n\tat org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) ~[spring-context-4.3.11.RELEASE.jar:4.3.11.RELEASE]\r\n\tat org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-4.3.11.RELEASE.jar:4.3.11.RELEASE]\r\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_151]\r\n\tat java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[na:1.8.0_151]\r\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_151]\r\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[na:1.8.0_151]\r\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_151]\r\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_151]\r\n\tat java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_151]\r\nCaused by: com.netflix.spinnaker.clouddriver.kubernetes.v2.op.job.KubectlJobExecutor$KubectlException: Failed to read [namespace] from : Unable to connect to the server: error executing access token command \"/usr/lib/google-cloud-sdk/bin/gcloud config config-helper --format=json\": err=fork/exec /usr/lib/google-cloud-sdk/bin/gcloud: no such file or directory output= stderr=\r\n\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/spinnaker/spinnaker/issues/comments/397320789","html_url":"https://github.com/spinnaker/spinnaker/issues/2900#issuecomment-397320789","issue_url":"https://api.github.com/repos/spinnaker/spinnaker/issues/2900","id":397320789,"node_id":"MDEyOklzc3VlQ29tbWVudDM5NzMyMDc4OQ==","user":{"login":"HirokiSakonju","id":4677344,"node_id":"MDQ6VXNlcjQ2NzczNDQ=","avatar_url":"https://avatars1.githubusercontent.com/u/4677344?v=4","gravatar_id":"","url":"https://api.github.com/users/HirokiSakonju","html_url":"https://github.com/HirokiSakonju","followers_url":"https://api.github.com/users/HirokiSakonju/followers","following_url":"https://api.github.com/users/HirokiSakonju/following{/other_user}","gists_url":"https://api.github.com/users/HirokiSakonju/gists{/gist_id}","starred_url":"https://api.github.com/users/HirokiSakonju/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HirokiSakonju/subscriptions","organizations_url":"https://api.github.com/users/HirokiSakonju/orgs","repos_url":"https://api.github.com/users/HirokiSakonju/repos","events_url":"https://api.github.com/users/HirokiSakonju/events{/privacy}","received_events_url":"https://api.github.com/users/HirokiSakonju/received_events","type":"User","site_admin":false},"created_at":"2018-06-14T14:42:43Z","updated_at":"2018-06-14T14:42:43Z","author_association":"NONE","body":"I'm sorry I didn't make it clear enough.\r\nI can access pod.\r\nBut since RBAC was enabled, it was not authorized to read the kubernetes resource from pod.\r\nSo I added permissions.\r\n\r\nOnce I added the permissions, I could read the resources of kubernetes as follows.\r\n```\r\n$ kubectl exec -it spin-clouddriver-6fcbf7cf7-s8b25 -n spinnaker kubectl get pod\r\nNAME                               READY     STATUS    RESTARTS   AGE\r\nspin-clouddriver-6fcbf7cf7-s8b25   0/1       Running   0          3m\r\nspin-deck-64c77858bc-sqvhv         1/1       Running   0          3m\r\nspin-echo-5b778876dd-bkx9x         1/1       Running   0          3m\r\nspin-front50-65c677f697-mgk7r      1/1       Running   0          3m\r\nspin-gate-7655445d46-zsx74         1/1       Running   0          3m\r\nspin-orca-b88b864d6-mdmkx          1/1       Running   0          3m\r\nspin-redis-6d49c9c5b9-c6wjj        1/1       Running   0          4m\r\nspin-rosco-8687b9db56-knbdj        1/1       Running   0          3m\r\n```\r\n\r\nHowever, `Readiness probe failed:` is still ongoing.\r\nAfter doing `hal deploy clean`, I executed`hal deploy apply` again, but it did not change."}}
{"actor":{"display_login":"HirokiSakonju"}}