{"action":"created","issue":{"url":"https://api.github.com/repos/Particular/NServiceBus.RavenDB/issues/140","repository_url":"https://api.github.com/repos/Particular/NServiceBus.RavenDB","labels_url":"https://api.github.com/repos/Particular/NServiceBus.RavenDB/issues/140/labels{/name}","comments_url":"https://api.github.com/repos/Particular/NServiceBus.RavenDB/issues/140/comments","events_url":"https://api.github.com/repos/Particular/NServiceBus.RavenDB/issues/140/events","html_url":"https://github.com/Particular/NServiceBus.RavenDB/issues/140","id":128767199,"number":140,"title":"RavenDB timeout query might prevent endpoint from shutting down","user":{"login":"SzymonPobiega","id":163318,"avatar_url":"https://avatars.githubusercontent.com/u/163318?v=3","gravatar_id":"","url":"https://api.github.com/users/SzymonPobiega","html_url":"https://github.com/SzymonPobiega","followers_url":"https://api.github.com/users/SzymonPobiega/followers","following_url":"https://api.github.com/users/SzymonPobiega/following{/other_user}","gists_url":"https://api.github.com/users/SzymonPobiega/gists{/gist_id}","starred_url":"https://api.github.com/users/SzymonPobiega/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SzymonPobiega/subscriptions","organizations_url":"https://api.github.com/users/SzymonPobiega/orgs","repos_url":"https://api.github.com/users/SzymonPobiega/repos","events_url":"https://api.github.com/users/SzymonPobiega/events{/privacy}","received_events_url":"https://api.github.com/users/SzymonPobiega/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/Particular/NServiceBus.RavenDB/labels/Tag:%20Core%20v6","name":"Tag: Core v6","color":"ffff00"},{"url":"https://api.github.com/repos/Particular/NServiceBus.RavenDB/labels/Type:%20Bug","name":"Type: Bug","color":"e10c02"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/Particular/NServiceBus.RavenDB/milestones/27","html_url":"https://github.com/Particular/NServiceBus.RavenDB/milestones/3.1.1","labels_url":"https://api.github.com/repos/Particular/NServiceBus.RavenDB/milestones/27/labels","id":1575419,"number":27,"title":"3.1.1","description":"","creator":{"login":"DavidBoike","id":427110,"avatar_url":"https://avatars.githubusercontent.com/u/427110?v=3","gravatar_id":"","url":"https://api.github.com/users/DavidBoike","html_url":"https://github.com/DavidBoike","followers_url":"https://api.github.com/users/DavidBoike/followers","following_url":"https://api.github.com/users/DavidBoike/following{/other_user}","gists_url":"https://api.github.com/users/DavidBoike/gists{/gist_id}","starred_url":"https://api.github.com/users/DavidBoike/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DavidBoike/subscriptions","organizations_url":"https://api.github.com/users/DavidBoike/orgs","repos_url":"https://api.github.com/users/DavidBoike/repos","events_url":"https://api.github.com/users/DavidBoike/events{/privacy}","received_events_url":"https://api.github.com/users/DavidBoike/received_events","type":"User","site_admin":false},"open_issues":2,"closed_issues":0,"state":"open","created_at":"2016-02-10T17:57:07Z","updated_at":"2016-02-11T07:45:11Z","due_on":"2016-02-12T06:00:00Z","closed_at":null},"comments":11,"created_at":"2016-01-26T09:15:02Z","updated_at":"2016-02-15T11:00:55Z","closed_at":null,"body":"### Symptoms\r\n\r\nEndpoint is unable to complete the shutdown procedure due to timeout poller being stuck in `GetNextChunk` query.\r\n\r\n### Description\r\n\r\nThe RavenDB implementation of `GetNextChunk` uses the stream API which can block forever if inserts outpace the reads. In our scenario we are sending out messages (doing I/O operations) inside the loop so if the queue is slower than raven inserts, the query will never return.\r\n\r\nHere's a snippet that shows the problem\r\n```\r\nusing System;\r\nusing System.Collections.Generic;\r\nusing System.Linq;\r\nusing System.Text;\r\nusing System.Threading;\r\nusing System.Threading.Tasks;\r\nusing Raven.Client.Document;\r\n\r\nnamespace RavenNeverendingStream\r\n{\r\n    class Program\r\n    {\r\n        static void Main(string[] args)\r\n        {\r\n            var store = new DocumentStore()\r\n            {\r\n                Url = \"http://localhost:8083\"\r\n            }.Initialize();\r\n\r\n            var producer = Task.Run(() =>\r\n            {\r\n                var i = 0;\r\n                while (true)\r\n                {\r\n                    using (var session = store.OpenSession(\"Test1\"))\r\n                    {\r\n                        session.Store(new Timeout {Number = i});\r\n                        session.SaveChanges();\r\n                        i++;\r\n                    }\r\n                    Thread.Sleep(150);\r\n                }\r\n            });\r\n\r\n            var consumer = Task.Run(() =>\r\n            {\r\n                while (true)\r\n                {\r\n                    try\r\n                    {\r\n                        Console.WriteLine(\"Starting new query.\");\r\n                        using (var session = store.OpenSession(\"Test1\"))\r\n                        {\r\n                            Console.WriteLine(\"Opening stream.\");\r\n                            var query = session.Query<Timeout>(\"SomeIndex\");\r\n                            var stream = session.Advanced.Stream(query);\r\n                            Console.WriteLine(\"Enumerating the stream.\");\r\n                            int j = 0;\r\n                            while (stream.MoveNext())\r\n                            {\r\n                                j++;\r\n                                //Console.WriteLine(stream.Current.Document.Id);\r\n                                session.Delete(stream.Current.Document.Id);\r\n                                Thread.Sleep(200);\r\n                            }\r\n                            Console.WriteLine(j);\r\n                        }\r\n                    }\r\n                    catch (Exception ex)\r\n                    {\r\n                        Console.WriteLine(ex);\r\n                    }\r\n                    \r\n                }\r\n            });\r\n            Console.ReadLine();\r\n        }\r\n    }\r\n\r\n    public class Timeout\r\n    {\r\n        public string Id { get; set; }\r\n        public int Number { get; set; }\r\n    }\r\n}\r\n```\r\n\r\n### Who's affected\r\n\r\nRavenDB persistence users who have endpoints with high timeout throughput.\r\n\r\n### Suggested resolution\r\n\r\nBound the stream query to not buffer more than a certain number of timeouts"},"comment":{"url":"https://api.github.com/repos/Particular/NServiceBus.RavenDB/issues/comments/184166819","html_url":"https://github.com/Particular/NServiceBus.RavenDB/issues/140#issuecomment-184166819","issue_url":"https://api.github.com/repos/Particular/NServiceBus.RavenDB/issues/140","id":184166819,"user":{"login":"andreasohlund","id":125028,"avatar_url":"https://avatars.githubusercontent.com/u/125028?v=3","gravatar_id":"","url":"https://api.github.com/users/andreasohlund","html_url":"https://github.com/andreasohlund","followers_url":"https://api.github.com/users/andreasohlund/followers","following_url":"https://api.github.com/users/andreasohlund/following{/other_user}","gists_url":"https://api.github.com/users/andreasohlund/gists{/gist_id}","starred_url":"https://api.github.com/users/andreasohlund/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andreasohlund/subscriptions","organizations_url":"https://api.github.com/users/andreasohlund/orgs","repos_url":"https://api.github.com/users/andreasohlund/repos","events_url":"https://api.github.com/users/andreasohlund/events{/privacy}","received_events_url":"https://api.github.com/users/andreasohlund/received_events","type":"User","site_admin":false},"created_at":"2016-02-15T11:00:55Z","updated_at":"2016-02-15T11:00:55Z","body":">I got it, the idea when using the streaming API is to avoid the pagination issue\r\n\r\nYes\r\n\r\n\r\nSince the number of documents found can be large I'd say a cap on number is better since a cap on time might still endup with to many documents to dispatch?"}}