{"action":"created","issue":{"url":"https://api.github.com/repos/microservices-demo/microservices-demo/issues/300","repository_url":"https://api.github.com/repos/microservices-demo/microservices-demo","labels_url":"https://api.github.com/repos/microservices-demo/microservices-demo/issues/300/labels{/name}","comments_url":"https://api.github.com/repos/microservices-demo/microservices-demo/issues/300/comments","events_url":"https://api.github.com/repos/microservices-demo/microservices-demo/issues/300/events","html_url":"https://github.com/microservices-demo/microservices-demo/issues/300","id":174896810,"number":300,"title":"Problems with kubernetes deployment","user":{"login":"deafRandom","id":21243919,"avatar_url":"https://avatars.githubusercontent.com/u/21243919?v=3","gravatar_id":"","url":"https://api.github.com/users/deafRandom","html_url":"https://github.com/deafRandom","followers_url":"https://api.github.com/users/deafRandom/followers","following_url":"https://api.github.com/users/deafRandom/following{/other_user}","gists_url":"https://api.github.com/users/deafRandom/gists{/gist_id}","starred_url":"https://api.github.com/users/deafRandom/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/deafRandom/subscriptions","organizations_url":"https://api.github.com/users/deafRandom/orgs","repos_url":"https://api.github.com/users/deafRandom/repos","events_url":"https://api.github.com/users/deafRandom/events{/privacy}","received_events_url":"https://api.github.com/users/deafRandom/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-09-03T13:22:12Z","updated_at":"2016-09-06T13:23:57Z","closed_at":"2016-09-06T13:23:57Z","body":"Hi, my goal is to deploy your demo, run some load tests and analyse the logs. I use kubernetes because it can be used with different cloud providers.\r\n\r\nI'm struggling with three things:\r\n1: Whenever I change the /etc/sysconfig/kubelet file to include `--network-plugin=cni --network-plugin-dir=/etc/cni/net.d` (as described in your [guide](https://github.com/microservices-demo/microservices-demo/blob/master/docs/create-staging-environment.md)), I encounter the following errors when executing the `sudo service kubelet restart` command:\r\n ```\r\n53s           53s             1       {kubelet ip-172-20-0-150.eu-west-1.compute.internal}\r\nNormal          InfraChanged            Pod infrastructure changed, it will be killed and re-created.\r\n 53s           53s             1       {kubelet ip-172-20-0-150.eu-west-1.compute.internal}\r\nWarning         FailedSync              Error syncing pod, skipping: failed to \"TeardownNetwork\" for \"kubernetes-dashboard-v1.1.1-244gu_kube-system\" with TeardownNetworkError: \"Failed to teardown network for pod \\\"6a0820e3-719b-11e6-963e-0a9631e405f3\\\" using network plugins \\\"cni\\\": unable to release IP address: 400 Bad Request: Delete: no addresses for df426f330dea8470128700978c40cd0c9a5cabab86f831748a35ee58343cf35b\\n\"\r\n\r\nError syncing pod, skipping: failed to \"SetupNetwork\" for \"catalogue-db-3749174200-f4wkt_default\" with SetupNetworkError: \"Failed to setup network for pod \\\"catalogue-db-3749174200-f4wkt_default(44ab7ad4-7022-11e6-aaa7-0ad1a2d3c31d)\\\" using network plugins \\\"cni\\\": could not find \\\".\\\" plugin; Skipping pod\" \r\n```\r\n\r\nI could only find one working solution and that is to reboot all minion EC2 instances. Unfortunately, a minion (m3.medium) takes about 10 minutes to be up and running again. Is there a faster way?\r\n\r\n2: My goal is to run the setup once and then save the state of the machines so I can start them again quickly whenever I need to. As a first step in that direction, I want to be able to stop and restart the master node on AWS (I shutdown the minions by setting min, max, desired in the auto scaling groups to 0). Unfortunately, every time I restart the master and type \"sudo weave launch\" I get the following error:\r\n```\r\n\"Could not determine address assignment type of docker0\r\nDevice \"docker0\" does not exist.\"\r\n```\r\nAs far as I understand, \"docker0\" is the default bridge in docker for Mac. On Linux the default name is \"cbr0\". When I run the whole deployment script from start I don't get this error, although the bridge is still cbr0. I checked this by typing:\r\n\r\n```\r\nsudo docker network inspect bridge\r\n[\r\n    {\r\n        \"Name\": \"bridge\",\r\n        \"Id\": \"4fdf51dc95d79cf2e29fdcd277d26230fdc34faab2e34ca0193dac1aa8ffbb0d\",\r\n        \"Scope\": \"local\",\r\n        \"Driver\": \"bridge\",\r\n        \"EnableIPv6\": false,\r\n        \"IPAM\": {\r\n            \"Driver\": \"default\",\r\n            \"Options\": null,\r\n            \"Config\": [\r\n                {\r\n                    \"Subnet\": \"10.246.0.0/24\",\r\n                    \"Gateway\": \"10.246.0.1\"\r\n                }\r\n            ]\r\n        },\r\n        \"Internal\": false,\r\n        \"Containers\": {},\r\n        \"Options\": {\r\n            \"com.docker.network.bridge.default_bridge\": \"true\",\r\n            \"com.docker.network.bridge.enable_icc\": \"true\",\r\n            \"com.docker.network.bridge.enable_ip_masquerade\": \"false\",\r\n            \"com.docker.network.bridge.host_binding_ipv4\": \"0.0.0.0\",\r\n            \"com.docker.network.bridge.name\": \"cbr0\",\r\n            \"com.docker.network.driver.mtu\": \"1500\"\r\n        },\r\n        \"Labels\": {}\r\n    }\r\n]\r\n```\r\n\r\nIs there any way to get weave to start again? You mention in your [docs](https://www.weave.works/documentation/net-latest-weavedns/net-latest-how-works-weavedns/) that I could use `sudo DOCKER_BRIDGE=someother weave launch` ~~but the front-end is not reachable that way, although all services are up~~. EDIT: I just noticed that after submitting this message the frontend was accessible again.\r\n\r\n\r\n3: This is not a weave specific problem. I want to use the logging apps (kibana, elasticsearch, fluentd) that come with the kubernetes deploy script. However, fluentd does not have the rights to read the log files stored in /var/log/containers/ :\r\n`/var/log/containers/elasticsearch-logging-v1-s3qoc_kube-system_elasticsearch-logging.log unreadable. It is excluded and would be examined next time.`\r\nDo you know of a way to fix that in the setup script?\r\n\r\nThanks a lot for your help!"},"comment":{"url":"https://api.github.com/repos/microservices-demo/microservices-demo/issues/comments/244948414","html_url":"https://github.com/microservices-demo/microservices-demo/issues/300#issuecomment-244948414","issue_url":"https://api.github.com/repos/microservices-demo/microservices-demo/issues/300","id":244948414,"user":{"login":"philwinder","id":8793723,"avatar_url":"https://avatars.githubusercontent.com/u/8793723?v=3","gravatar_id":"","url":"https://api.github.com/users/philwinder","html_url":"https://github.com/philwinder","followers_url":"https://api.github.com/users/philwinder/followers","following_url":"https://api.github.com/users/philwinder/following{/other_user}","gists_url":"https://api.github.com/users/philwinder/gists{/gist_id}","starred_url":"https://api.github.com/users/philwinder/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/philwinder/subscriptions","organizations_url":"https://api.github.com/users/philwinder/orgs","repos_url":"https://api.github.com/users/philwinder/repos","events_url":"https://api.github.com/users/philwinder/events{/privacy}","received_events_url":"https://api.github.com/users/philwinder/received_events","type":"User","site_admin":false},"created_at":"2016-09-06T13:23:57Z","updated_at":"2016-09-06T13:23:57Z","body":"Hi deafRandom, I'll try and answer some of your issues.\r\n\r\n1. Don't start the application before setting up the cluster. Setup the cluster, then start the application. I've also had issues when the app is running and you have an unconfigured node. This is more of a k8s issue than a weave demo one.\r\n\r\n2. That is a weave net issue. I've also had that problem and could only fix with a restart. I think it is something to do with some assumption about the name of the network adaptor. After restarting, it doesn't have the same name and weave net can't find it.\r\n\r\n3. This is a k8s issue. I haven't come across this, so I can't help unfortunately.\r\n\r\nIn summary:\r\n1. Setup the cluster first. If you continue to have issues, speak to the k8s guys.\r\n2. Submit issues to https://github.com/weaveworks/weave\r\n3. Search for/ask k8s guys.\r\n\r\nIt is annoying when things don't work as advertised, but most of the time the issues are caused by third party components. I have recently followed the documentation to create a staging environment and it does work. However, I started seeing issues like you once I started messing around with the working setup. My issues were a combination of k8s and docker problems.\r\n\r\nThanks for taking the time to try this and I hope you have another go. Let us know if you have any more problems."}}
{"actor":{"display_login":"philwinder"}}