{"action":"created","issue":{"url":"https://api.github.com/repos/spring-cloud/spring-cloud-gcp/issues/1005","repository_url":"https://api.github.com/repos/spring-cloud/spring-cloud-gcp","labels_url":"https://api.github.com/repos/spring-cloud/spring-cloud-gcp/issues/1005/labels{/name}","comments_url":"https://api.github.com/repos/spring-cloud/spring-cloud-gcp/issues/1005/comments","events_url":"https://api.github.com/repos/spring-cloud/spring-cloud-gcp/issues/1005/events","html_url":"https://github.com/spring-cloud/spring-cloud-gcp/issues/1005","id":358515297,"node_id":"MDU6SXNzdWUzNTg1MTUyOTc=","number":1005,"title":"[PubSub] Messages stuck in the queue and not being pulled by listener","user":{"login":"ramzimaalej","id":1727621,"node_id":"MDQ6VXNlcjE3Mjc2MjE=","avatar_url":"https://avatars0.githubusercontent.com/u/1727621?v=4","gravatar_id":"","url":"https://api.github.com/users/ramzimaalej","html_url":"https://github.com/ramzimaalej","followers_url":"https://api.github.com/users/ramzimaalej/followers","following_url":"https://api.github.com/users/ramzimaalej/following{/other_user}","gists_url":"https://api.github.com/users/ramzimaalej/gists{/gist_id}","starred_url":"https://api.github.com/users/ramzimaalej/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ramzimaalej/subscriptions","organizations_url":"https://api.github.com/users/ramzimaalej/orgs","repos_url":"https://api.github.com/users/ramzimaalej/repos","events_url":"https://api.github.com/users/ramzimaalej/events{/privacy}","received_events_url":"https://api.github.com/users/ramzimaalej/received_events","type":"User","site_admin":false},"labels":[{"id":925670225,"node_id":"MDU6TGFiZWw5MjU2NzAyMjU=","url":"https://api.github.com/repos/spring-cloud/spring-cloud-gcp/labels/pubsub","name":"pubsub","color":"efd021","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":32,"created_at":"2018-09-10T08:19:25Z","updated_at":"2018-09-13T17:41:20Z","closed_at":null,"author_association":"NONE","body":"I have a spring boot application (v 2.0.3.RELEASE) that uses spring-cloud-pubsub (1.0.0.RELEASE) and it's deployed on GKE. I noticed that after some period of inactivity the listener is no longer receiving messages from pubsub and messages get stuck in the queue for a long time (sometimes more than an a hour). I tried increasing the parallel-pull-count but no luck. As soon as I deploy the service the messages get acknowledged then they start piling up in the queue. I have researched this problem and some people think that the connection is dropped by gke due to inactivity. I have also tried spring-cloud-stream-pubsub but I got the same behavior. any ideas on how to get around this?\r\n\r\n<img width=\"653\" alt=\"screen shot 2018-09-10 at 1 15 40 am\" src=\"https://user-images.githubusercontent.com/1727621/45285100-22b2ba80-b497-11e8-89df-d8a42b8d95f1.png\">\r\n "},"comment":{"url":"https://api.github.com/repos/spring-cloud/spring-cloud-gcp/issues/comments/421091397","html_url":"https://github.com/spring-cloud/spring-cloud-gcp/issues/1005#issuecomment-421091397","issue_url":"https://api.github.com/repos/spring-cloud/spring-cloud-gcp/issues/1005","id":421091397,"node_id":"MDEyOklzc3VlQ29tbWVudDQyMTA5MTM5Nw==","user":{"login":"qlodhi-clearlabs","id":36972852,"node_id":"MDQ6VXNlcjM2OTcyODUy","avatar_url":"https://avatars1.githubusercontent.com/u/36972852?v=4","gravatar_id":"","url":"https://api.github.com/users/qlodhi-clearlabs","html_url":"https://github.com/qlodhi-clearlabs","followers_url":"https://api.github.com/users/qlodhi-clearlabs/followers","following_url":"https://api.github.com/users/qlodhi-clearlabs/following{/other_user}","gists_url":"https://api.github.com/users/qlodhi-clearlabs/gists{/gist_id}","starred_url":"https://api.github.com/users/qlodhi-clearlabs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/qlodhi-clearlabs/subscriptions","organizations_url":"https://api.github.com/users/qlodhi-clearlabs/orgs","repos_url":"https://api.github.com/users/qlodhi-clearlabs/repos","events_url":"https://api.github.com/users/qlodhi-clearlabs/events{/privacy}","received_events_url":"https://api.github.com/users/qlodhi-clearlabs/received_events","type":"User","site_admin":false},"created_at":"2018-09-13T17:41:20Z","updated_at":"2018-09-13T17:41:20Z","author_association":"NONE","body":"So, Google came back saying that the issue we faced \"was due to an internal problem\" and we should continue to do things as we were and report any issues if it appears again. Our original code (using `spring-gcp-pubsub`) is now receiving messages after running for ~12 hours so it appears that underlying problem is fixed on pubsub side. But for now, we will move to using `google-cloud-pubsub` directly as we've already migrated half of our environments in this process of troubleshooting.\r\n\r\nIt is worry-some that code base using `spring-gcp-pubsub` got stuck while direct subscriber using `google-cloud-pubsub` continued to work even when there was an underlying problem because that problem can appear again anytime, out of our control. @artembilan good suggestion, I'll continue to dig more in that direction."}}
{"actor":{"display_login":"qlodhi-clearlabs"}}