{"action":"created","issue":{"url":"https://api.github.com/repos/kubeflow/kubeflow/issues/1389","repository_url":"https://api.github.com/repos/kubeflow/kubeflow","labels_url":"https://api.github.com/repos/kubeflow/kubeflow/issues/1389/labels{/name}","comments_url":"https://api.github.com/repos/kubeflow/kubeflow/issues/1389/comments","events_url":"https://api.github.com/repos/kubeflow/kubeflow/issues/1389/events","html_url":"https://github.com/kubeflow/kubeflow/issues/1389","id":351994871,"node_id":"MDU6SXNzdWUzNTE5OTQ4NzE=","number":1389,"title":"GKE Deploy fails in European Zone as no NVIDIA GPUs available","user":{"login":"cliveseldon","id":10563075,"node_id":"MDQ6VXNlcjEwNTYzMDc1","avatar_url":"https://avatars2.githubusercontent.com/u/10563075?v=4","gravatar_id":"","url":"https://api.github.com/users/cliveseldon","html_url":"https://github.com/cliveseldon","followers_url":"https://api.github.com/users/cliveseldon/followers","following_url":"https://api.github.com/users/cliveseldon/following{/other_user}","gists_url":"https://api.github.com/users/cliveseldon/gists{/gist_id}","starred_url":"https://api.github.com/users/cliveseldon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cliveseldon/subscriptions","organizations_url":"https://api.github.com/users/cliveseldon/orgs","repos_url":"https://api.github.com/users/cliveseldon/repos","events_url":"https://api.github.com/users/cliveseldon/events{/privacy}","received_events_url":"https://api.github.com/users/cliveseldon/received_events","type":"User","site_admin":false},"labels":[{"id":765839028,"node_id":"MDU6TGFiZWw3NjU4MzkwMjg=","url":"https://api.github.com/repos/kubeflow/kubeflow/labels/community/question","name":"community/question","color":"ecfc15","default":false},{"id":810949496,"node_id":"MDU6TGFiZWw4MTA5NDk0OTY=","url":"https://api.github.com/repos/kubeflow/kubeflow/labels/platform/gcp","name":"platform/gcp","color":"2515fc","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-08-20T05:57:22Z","updated_at":"2018-11-19T04:18:05Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"\r\nThe core error seems to be\r\n\r\n```\r\ntype \\\"nvidia-tesla-k80\\\" does not exist in zone europe-west4-b\\ngoroutine 131398674\r\n```\r\nFull trace:\r\n\r\n```\r\n+ gcloud deployment-manager --project=kubeflow-testing-213813 deployments create kubeflow-testing --config=cluster-kubeflow.yaml\r\nThe fingerprint of the deployment is uhkZZ-qMmAJDKd0-Ya9F_w==\r\nWaiting for create [operation-1534744332224-573d782be4600-1322585d-8ace48b9]...\r\n........................................................................................................failed.\r\nERROR: (gcloud.deployment-manager.deployments.create) Error in Operation [operation-1534744332224-573d782be4600-1322585d-8ace48b9]: errors:\r\n- code: RESOURCE_ERROR\r\n  location: /deployments/kubeflow-testing/resources/kubeflow-testing-gpu-pool-v1\r\n  message: '{\"ResourceType\":\"container.v1.nodePool\",\"ResourceErrorCode\":\"400\",\"ResourceErrorMessage\":{\"code\":400,\"message\":\"Accelerator\r\n    type \\\"nvidia-tesla-k80\\\" does not exist in zone europe-west4-b\\ngoroutine 131398674\r\n    [running]:\\nruntime/debug.Stack(0x5436440, 0x0, 0x0)\\n\\tthird_party/go/gc/src/runtime/debug/stack.go:24\r\n    +0xa7\\ngoogle3/cloud/kubernetes/engine/common/errdesc.(*GKEErrorDescriptor).createErr(0x543f500,\r\n    0x2d)\\n\\tcloud/kubernetes/engine/common/error_desc.go:199 +0x26\\ngoogle3/cloud/kubernetes/engine/common/errdesc.(*GKEErrorDescriptor).WithMsg(0x543f500,\r\n    0x2e12b4b, 0x2d, 0xc018107ce8, 0x2, 0x2, 0x23, 0xc018107d08)\\n\\tcloud/kubernetes/engine/common/error_desc.go:142\r\n    +0x99\\ngoogle3/cloud/kubernetes/engine/common/clients/gce/gce.(*authedClient).validateAcceleratorAvailability(0xc00b63cd20,\r\n    0x310c800, 0xc010bf92f0, 0xc006c5d4a0, 0xc004ff1f60, 0x17, 0xc006c7fde0, 0xe,\r\n    0xc007a63538, 0x1, ...)\\n\\tcloud/kubernetes/engine/common/clients/gce/gce.go:3844\r\n    +0x35f\\ngoogle3/cloud/kubernetes/engine/common/clients/gce/gce.(*authedClient).validateCommon.func1(0x574b61,\r\n    0x54c299)\\n\\tcloud/kubernetes/engine/common/clients/gce/gce.go:2864 +0x6b3\\ngoogle3/cloud/kubernetes/engine/common/errors.CollectFns.func1(0xc00f6f1c20,\r\n    0xc00b605ce0)\\n\\tcloud/kubernetes/engine/common/errors.go:160 +0x27\\ncreated by\r\n    google3/cloud/kubernetes/engine/common/errors.CollectFns\\n\\tcloud/kubernetes/engine/common/errors.go:160\r\n    +0x82\\n.\",\"status\":\"INVALID_ARGUMENT\",\"statusMessage\":\"Bad Request\",\"requestPath\":\"https://container.googleapis.com/v1/projects/kubeflow-testing-213813/zones/europe-west4-b/clusters/kubeflow-testing/nodePools\",\"httpMethod\":\"POST\"}}'\r\n```"},"comment":{"url":"https://api.github.com/repos/kubeflow/kubeflow/issues/comments/439767145","html_url":"https://github.com/kubeflow/kubeflow/issues/1389#issuecomment-439767145","issue_url":"https://api.github.com/repos/kubeflow/kubeflow/issues/1389","id":439767145,"node_id":"MDEyOklzc3VlQ29tbWVudDQzOTc2NzE0NQ==","user":{"login":"jlewi","id":777219,"node_id":"MDQ6VXNlcjc3NzIxOQ==","avatar_url":"https://avatars3.githubusercontent.com/u/777219?v=4","gravatar_id":"","url":"https://api.github.com/users/jlewi","html_url":"https://github.com/jlewi","followers_url":"https://api.github.com/users/jlewi/followers","following_url":"https://api.github.com/users/jlewi/following{/other_user}","gists_url":"https://api.github.com/users/jlewi/gists{/gist_id}","starred_url":"https://api.github.com/users/jlewi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jlewi/subscriptions","organizations_url":"https://api.github.com/users/jlewi/orgs","repos_url":"https://api.github.com/users/jlewi/repos","events_url":"https://api.github.com/users/jlewi/events{/privacy}","received_events_url":"https://api.github.com/users/jlewi/received_events","type":"User","site_admin":false},"created_at":"2018-11-19T04:18:05Z","updated_at":"2018-11-19T04:18:05Z","author_association":"CONTRIBUTOR","body":"Thanks @ofaz \r\n\r\nThis should be fixed at head. \r\nhttps://github.com/kubeflow/kubeflow/blob/41050ef854948600ef7f922f2a3c295b60b1f7c0/deployment/gke/deployment_manager_configs/cluster.jinja#L115\r\n\r\nWe no longer add the GPU node pool if max gpu nodes is 0.\r\n\r\nThe change isn't on the 0.3 branch\r\nhttps://github.com/kubeflow/kubeflow/blob/dddba26893846041b7cee25ee6121e02a04bd503/scripts/gke/deployment_manager_configs/cluster.jinja#L116\r\n"}}
{"actor":{"display_login":"jlewi"}}