{"action":"created","issue":{"url":"https://api.github.com/repos/dotnet/orleans/issues/1245","labels_url":"https://api.github.com/repos/dotnet/orleans/issues/1245/labels{/name}","comments_url":"https://api.github.com/repos/dotnet/orleans/issues/1245/comments","events_url":"https://api.github.com/repos/dotnet/orleans/issues/1245/events","html_url":"https://github.com/dotnet/orleans/pull/1245","id":125300175,"number":1245,"title":"PooledQueueCache","user":{"login":"jason-bragg","id":10675494,"avatar_url":"https://avatars.githubusercontent.com/u/10675494?v=3","gravatar_id":"","url":"https://api.github.com/users/jason-bragg","html_url":"https://github.com/jason-bragg","followers_url":"https://api.github.com/users/jason-bragg/followers","following_url":"https://api.github.com/users/jason-bragg/following{/other_user}","gists_url":"https://api.github.com/users/jason-bragg/gists{/gist_id}","starred_url":"https://api.github.com/users/jason-bragg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jason-bragg/subscriptions","organizations_url":"https://api.github.com/users/jason-bragg/orgs","repos_url":"https://api.github.com/users/jason-bragg/repos","events_url":"https://api.github.com/users/jason-bragg/events{/privacy}","received_events_url":"https://api.github.com/users/jason-bragg/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/dotnet/orleans/labels/cla-already-signed","name":"cla-already-signed","color":"009800"},{"url":"https://api.github.com/repos/dotnet/orleans/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":{"login":"gabikliot","id":10171207,"avatar_url":"https://avatars.githubusercontent.com/u/10171207?v=3","gravatar_id":"","url":"https://api.github.com/users/gabikliot","html_url":"https://github.com/gabikliot","followers_url":"https://api.github.com/users/gabikliot/followers","following_url":"https://api.github.com/users/gabikliot/following{/other_user}","gists_url":"https://api.github.com/users/gabikliot/gists{/gist_id}","starred_url":"https://api.github.com/users/gabikliot/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gabikliot/subscriptions","organizations_url":"https://api.github.com/users/gabikliot/orgs","repos_url":"https://api.github.com/users/gabikliot/repos","events_url":"https://api.github.com/users/gabikliot/events{/privacy}","received_events_url":"https://api.github.com/users/gabikliot/received_events","type":"User","site_admin":false},"milestone":null,"comments":8,"created_at":"2016-01-07T01:11:47Z","updated_at":"2016-01-13T20:14:35Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/dotnet/orleans/pulls/1245","html_url":"https://github.com/dotnet/orleans/pull/1245","diff_url":"https://github.com/dotnet/orleans/pull/1245.diff","patch_url":"https://github.com/dotnet/orleans/pull/1245.patch"},"body":"This is the initial prototype of a pooled cache for the recoverable `EventHubStreamProvider`.\r\n\r\nSee: Recoverable Event Hub Persistent Stream Provider #1096\r\nThis is approximately what is described in \"2. Introduce BigQueueCache...\"\r\n\r\nAt this point, this queue cache is mainly a set of components that can be assembled into a fast in memory cache.  Further development will work to wrap and simplify these components.\r\n\r\nThe main goal of this cache is to support the caching of high throughput queues (~2k+ messages / queue / second) for an extended period of time (~30 minutes of data) amounting to gigs worth of data with minimal performance degradation due to garbage collection.\r\n\r\nUnlike the simple queue cache, this queue purges data when cache capacity is reached, regardless of whether the events have been delivered or not.  The reasons for this is that it's intended for fast recoverable real time processing, so if processing gets too far behind (~15-30 minutes) it's preferable to drop those events and process new events in a timely manner than to fall further and further behind.\r\n\r\nThis implementation currently has no mechanism for regulating backpressure from stream processing delays.\r\n\r\nSome components this implementation includes:\r\n- A basic object pool\r\n- A fixed size object pool that will trigger a purge of resources when it's capacity is reached.\r\n- A fixed size buffer that allocates blocks of data used to cached serialized message payloads\r\n- A block of tightly packed message metadata structures for cached messages.\r\n\r\nTesting:\r\n- Minimal BVT tests have been added - more are needed.\r\n- GeneratorStreamProvider has been updated to use the PooledQueueCache.  It's test now exersize this code."},"comment":{"url":"https://api.github.com/repos/dotnet/orleans/issues/comments/171419947","html_url":"https://github.com/dotnet/orleans/pull/1245#issuecomment-171419947","issue_url":"https://api.github.com/repos/dotnet/orleans/issues/1245","id":171419947,"user":{"login":"amamh","id":2524201,"avatar_url":"https://avatars.githubusercontent.com/u/2524201?v=3","gravatar_id":"","url":"https://api.github.com/users/amamh","html_url":"https://github.com/amamh","followers_url":"https://api.github.com/users/amamh/followers","following_url":"https://api.github.com/users/amamh/following{/other_user}","gists_url":"https://api.github.com/users/amamh/gists{/gist_id}","starred_url":"https://api.github.com/users/amamh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amamh/subscriptions","organizations_url":"https://api.github.com/users/amamh/orgs","repos_url":"https://api.github.com/users/amamh/repos","events_url":"https://api.github.com/users/amamh/events{/privacy}","received_events_url":"https://api.github.com/users/amamh/received_events","type":"User","site_admin":false},"created_at":"2016-01-13T20:14:34Z","updated_at":"2016-01-13T20:14:34Z","body":"@jason-bragg \r\n\r\nThanks, that's very interesting. I really appreciate your taking the time to explain this :)\r\n\r\nI have some questions, if you don't mind:\r\n\r\n>  this pooled cache allocates memory and never releases it\r\n\r\nDoes that mean I cannot leave the server running for days? if the memory is never released, **at some point that will cause an issue**, isn't it?\r\n\r\nI have made a persistent stream using your cache and another one using a simple linked list cache which also never purges. I then wrote a **simple benchmark test where I send 1 million int messages on one stream and measure how long it takes to send/receive every 10k messages**.\r\n\r\nThe one with the [simple cache](https://github.com/amamh/orleans-PipeStreamProvider) performs 6 times faster when writing/sending and about 30% faster when reading/receiving than the [one with the pooled cache](https://github.com/amamh/orleans-FastPipeStreamProvider).\r\n\r\nI'm not expecting you to look into these two project, but maybe you can suggest **a test scenario where the pooled cache should perform better**?"}}