{"action":"opened","issue":{"url":"https://api.github.com/repos/containous/traefik/issues/3237","repository_url":"https://api.github.com/repos/containous/traefik","labels_url":"https://api.github.com/repos/containous/traefik/issues/3237/labels{/name}","comments_url":"https://api.github.com/repos/containous/traefik/issues/3237/comments","events_url":"https://api.github.com/repos/containous/traefik/issues/3237/events","html_url":"https://github.com/containous/traefik/issues/3237","id":317998028,"number":3237,"title":"Sporadic 502 response only when running through traefik","user":{"login":"koliyo","id":770348,"avatar_url":"https://avatars1.githubusercontent.com/u/770348?v=4","gravatar_id":"","url":"https://api.github.com/users/koliyo","html_url":"https://github.com/koliyo","followers_url":"https://api.github.com/users/koliyo/followers","following_url":"https://api.github.com/users/koliyo/following{/other_user}","gists_url":"https://api.github.com/users/koliyo/gists{/gist_id}","starred_url":"https://api.github.com/users/koliyo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/koliyo/subscriptions","organizations_url":"https://api.github.com/users/koliyo/orgs","repos_url":"https://api.github.com/users/koliyo/repos","events_url":"https://api.github.com/users/koliyo/events{/privacy}","received_events_url":"https://api.github.com/users/koliyo/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-04-26T12:01:30Z","updated_at":"2018-04-26T12:01:30Z","closed_at":null,"author_association":"NONE","body":"\r\n### Do you want to request a *feature* or report a *bug*?\r\nBug\r\n\r\n### What did you do?\r\n\r\nI have a graphql API running on NodeJS using Apollo and Express, with traefik in front.\r\n\r\nWhen proxying through traefik I get sporadic 502 responses that I have not been able to resolve.\r\n\r\nI does _never_ happen when I bypass the proxy and connect directly to the backend node server.\r\n\r\nI am running all tests locally on my dev machine.\r\n\r\nMy first attempt to force the error was load testing with the [locust](locust.io) framework. However, even when sending large amounts of request through the proxy I was unable to replicate the error. It only happens when I use the frontend application in the browser.\r\n\r\nAfter reading [this](https://github.com/vulcand/oxy/issues/112) oxy issue I started suspecting cancelled connections.\r\n\r\nI added a custom HTTP header with a UUID to be able to trace all requests, which I print on the backend.\r\n```js\r\n  app.use((req, res, next) => {\r\n    const id = req.headers['x-request-id'];\r\n    if (id) {\r\n      console.log(`Request id: ${id}`);\r\n    }\r\n    next();\r\n  });\r\n\r\n```\r\n\r\nThen I also added the following event listener to the express server to [detect cancelled requests](https://stackoverflow.com/questions/35198208/handling-cancelled-request-with-express-node-js-and-angular)\r\n```js\r\n  app.use((req, res, next) => {\r\n    req.connection.on('close', () => {\r\n      const id = req.headers['x-request-id'];\r\n      console.log(`Cancelled request ${id}`);\r\n    });\r\n\r\n    next();\r\n  });\r\n```\r\n\r\n\r\nWhat I can see is that I _do_ get cancelled requests when running the application in the browser, and at some point i get a `502` response from traefik. And in the traefik log this is \r\n\r\n```\r\nDEBU[2018-04-26T13:43:51+02:00] vulcand/oxy/forward/http: Round trip: http://localhost:6543, code: 502, Length: 11, duration: 66.352475ms \r\n```\r\n\r\nAnd the nodejs backend log looks something like this:\r\n\r\n```\r\n...\r\nRequest id: 7455804b-490a-4361-98e5-43d12bf4aca8\r\nRequest id: 737f8d9d-3300-461b-858b-07006582a937\r\nPOST /graphql 200 83.542 ms - 310\r\nPOST /graphql 200 16.441 ms - 682\r\nRequest id: 096e0e39-90e6-475c-b8ad-0aa2dfd2e345\r\nPOST /graphql 200 5.338 ms - 163\r\nRequest id: 69f17cb2-cdf1-4db5-a9f5-08e46d795892\r\nRequest id: 50d3aec6-5cda-4e8b-ac0e-a30a57fa94c9\r\nPOST /graphql 200 58.596 ms - 310\r\nPOST /graphql 200 15.526 ms - 682\r\nRequest id: 1d051f3a-7d80-464b-b50f-6d8e733d1940\r\n<------------- Here I get the 502\r\nCancelled request 2e0a8e14-9880-46e7-8e51-ad528d55a81d\r\nCancelled request b9489e71-7fc5-4f1c-b30a-668aac4652f9\r\nCancelled request 249c529c-b9cb-4b48-a491-8e38a7ee50d8\r\nCancelled request a5be4a66-9d43-4e30-a92d-862b355399a0\r\nCancelled request 3721fe71-fe18-4389-812a-a90cc2f4f0f1\r\nCancelled request 71b74750-8078-471e-91b8-a8119e5db797\r\nCancelled request 34fb6b91-9fa5-4d68-92da-c267089f5910\r\nCancelled request 692770b1-61c3-49c2-8309-8e7be629dca1\r\nCancelled request 05790579-8290-4787-a7b7-82596ad24520\r\nCancelled request c8edcc39-30c7-4812-941c-a1899298acf7\r\nCancelled request 2ba9e715-ab7c-48ee-9d35-b5609179de6e\r\nCancelled request b34f4725-665f-4b27-b3e1-cefec20c2ade\r\nCancelled request 04bd3718-f6aa-4318-a469-fa3e17f54a20\r\nCancelled request 4aedc60c-269a-420c-b083-1ea8f2e3243e\r\nCancelled request 25be7334-43f9-4135-9537-36b0e36e698c\r\nCancelled request 47bc1f9f-55c7-4f31-9957-7f0ad4285314\r\nCancelled request bae3237c-efc8-4831-8260-6edbcedef28f\r\nCancelled request 54685ecb-4d34-4698-b956-d0602b74a2e4\r\nCancelled request 965f6ff2-da91-423c-a8e4-c2f4252f25fc\r\nCancelled request 95c77d5c-230d-4875-8b25-fc0673c8e595\r\nCancelled request 01658960-4627-42f8-a496-d29408a9579b\r\nCancelled request 38221ac3-47ed-42f2-a56e-31deacdbfd62\r\nCancelled request e73bec6b-744c-47bc-b001-0d914f03e976\r\nCancelled request 73fade75-a943-45df-8b21-f8c50a480170\r\nCancelled request 02688ad9-e947-415f-b70c-3cda16c50cf2\r\nCancelled request 5d7d26c2-8c69-4083-a2d3-f0e1ae23bd0f\r\nCancelled request f81a0258-085d-462f-9fcb-8a8b47918d04\r\n...\r\n```\r\n\r\nThe failed request that gets a 502 response in the browser never reach the node server backend.\r\n\r\nI get a whole bunch of canceled request after the 502 occurs. These request IDs have been successfully served by the nodejs application at an earlier point.\r\n\r\nThe canceling of the request seem to indicate some kind of connection leak? Or maybe just a sideffect of having chrome developer tools open?\r\n\r\nAnyway I never get any error response when bypassing the traefik instance. \r\n\r\nAs the [oxy issue](https://github.com/vulcand/oxy/issues/112) describes, if I just could get some other response than 502 for cancelled requests I could handle this better on the client side.\r\n\r\n\r\n\r\n### Output of `traefik version`: (_What version of Traefik are you using?_)\r\nGet the problem with the docker release as well as my homebrew install\r\n\r\nHomebrew traefik version:\r\n```\r\nVersion:      dev\r\nCodename:     cheddar\r\nGo version:   go1.10\r\nBuilt:        I don't remember exactly\r\nOS/Arch:      darwin/amd64\r\n```\r\n\r\nDocker traefik version:\r\n```\r\nVersion:      v1.5.2\r\nCodename:     cancoillotte\r\nGo version:   go1.9.4\r\nBuilt:        2018-02-12_10:56:31AM\r\nOS/Arch:      linux/amd64\r\n```\r\n\r\n### What is your environment & configuration (arguments, toml, provider, platform, ...)?\r\n\r\n```toml\r\ndebug = true\r\n\r\nlogLevel = \"DEBUG\"\r\ndefaultEntryPoints = [\"http\"]\r\n\r\n[entryPoints]\r\n  [entryPoints.ping]\r\n  address = \":8082\"\r\n\r\n  [entryPoints.api]\r\n  address = \":8080\"\r\n\r\n  [entryPoints.http]\r\n  address = \":80\"\r\n\r\n[retry]\r\n\r\n[ping]\r\nentryPoint = \"ping\"\r\n\r\n[api]\r\nentryPoint = \"api\"\r\n  [api.statistics]\r\n\r\n[file]\r\n[backends]\r\n  [backends.bct]\r\n    [backends.bct.servers]\r\n      [backends.bct.servers.server0]\r\n        # url = \"http://docker.for.mac.host.internal:6543\"\r\n        url = \"http://localhost:6543\"\r\n\r\n\r\n[frontends]\r\n  [frontends.bct]\r\n    entryPoints = [\"http\"]\r\n    backend = \"bct\"\r\n\r\n[docker]\r\n  endpoint = \"unix:///var/run/docker.sock\"\r\n  # domain = \"docker.for.mac.host.internal\"\r\n  domain = \"localhost\"\r\n  watch = true\r\n  exposedbydefault = false\r\n\r\n```\r\n<!--\r\nAdd more configuration information here.\r\n-->\r\n\r\n"}}
{"actor":{"display_login":"koliyo"}}