{"action":"created","issue":{"url":"https://api.github.com/repos/fabiolb/fabio/issues/481","repository_url":"https://api.github.com/repos/fabiolb/fabio","labels_url":"https://api.github.com/repos/fabiolb/fabio/issues/481/labels{/name}","comments_url":"https://api.github.com/repos/fabiolb/fabio/issues/481/comments","events_url":"https://api.github.com/repos/fabiolb/fabio/issues/481/events","html_url":"https://github.com/fabiolb/fabio/issues/481","id":310485752,"number":481,"title":"Fabio Has Stale Routes","user":{"login":"codyja","id":23107237,"avatar_url":"https://avatars2.githubusercontent.com/u/23107237?v=4","gravatar_id":"","url":"https://api.github.com/users/codyja","html_url":"https://github.com/codyja","followers_url":"https://api.github.com/users/codyja/followers","following_url":"https://api.github.com/users/codyja/following{/other_user}","gists_url":"https://api.github.com/users/codyja/gists{/gist_id}","starred_url":"https://api.github.com/users/codyja/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/codyja/subscriptions","organizations_url":"https://api.github.com/users/codyja/orgs","repos_url":"https://api.github.com/users/codyja/repos","events_url":"https://api.github.com/users/codyja/events{/privacy}","received_events_url":"https://api.github.com/users/codyja/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2018-04-02T13:32:35Z","updated_at":"2018-04-02T19:39:52Z","closed_at":null,"author_association":"NONE","body":"Hi,\r\nWe've experienced and issue twice where a single fabio container in a group of fabios (running under nomad) happens to have old stale routes. Fabios is determinting routes to add to it's route table by connecting to the local Consul agent on the host, and looking for the default urlprefix- tag. The behavior I saw was some of our traffic was experiencing 502 errors. When traffic would get sent to the problematic fabio container, it would then try to deliver traffic to a container:port that didn't exist in the nomad cluster anymore. If I looked at /routes on this fabio, it didn't have the same routes as the other 9 or so fabios in this node class. Restarting the container resolves the immediate issue. We're running fabio 1.5.2.\r\n\r\nAny ideas what may cause this? Any ideas under what condition fabio may not refresh it's content from Consul? Thanks for any information. \r\n\r\n"},"comment":{"url":"https://api.github.com/repos/fabiolb/fabio/issues/comments/378021713","html_url":"https://github.com/fabiolb/fabio/issues/481#issuecomment-378021713","issue_url":"https://api.github.com/repos/fabiolb/fabio/issues/481","id":378021713,"user":{"login":"magiconair","id":229005,"avatar_url":"https://avatars3.githubusercontent.com/u/229005?v=4","gravatar_id":"","url":"https://api.github.com/users/magiconair","html_url":"https://github.com/magiconair","followers_url":"https://api.github.com/users/magiconair/followers","following_url":"https://api.github.com/users/magiconair/following{/other_user}","gists_url":"https://api.github.com/users/magiconair/gists{/gist_id}","starred_url":"https://api.github.com/users/magiconair/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/magiconair/subscriptions","organizations_url":"https://api.github.com/users/magiconair/orgs","repos_url":"https://api.github.com/users/magiconair/repos","events_url":"https://api.github.com/users/magiconair/events{/privacy}","received_events_url":"https://api.github.com/users/magiconair/received_events","type":"User","site_admin":false},"created_at":"2018-04-02T19:39:52Z","updated_at":"2018-04-02T19:39:52Z","author_association":"OWNER","body":"I meant the health data from Consul, https://www.consul.io/api/health.html#list-checks-in-state\r\n\r\nFabio runs this query to get the current health data for all services from Consul and then filters out the healthy ones. \r\n\r\n```\r\ncurl http://localhost:8500/v1/health/state/any\r\n```\r\n\r\nTwo observations:\r\n\r\n1. Why is your health data changing all the time? This looks like either a flapping service or a volatile health data output, e.g. a timestamp or process id in the output of the script.\r\n2. I'm not sure why the Manual config is changing as well. Do you have a script that writes to the `fabio/config` KV key?\r\n\r\nAgain, fabio pulls its data from consul and all fabios are doing the same thing since they rely on Consul to provide the consistency. Your problem is way more likely in your Consul setup than in fabio. "}}
{"actor":{"display_login":"magiconair"}}