{"action":"created","issue":{"url":"https://api.github.com/repos/farsightsec/fstrm/issues/42","repository_url":"https://api.github.com/repos/farsightsec/fstrm","labels_url":"https://api.github.com/repos/farsightsec/fstrm/issues/42/labels{/name}","comments_url":"https://api.github.com/repos/farsightsec/fstrm/issues/42/comments","events_url":"https://api.github.com/repos/farsightsec/fstrm/issues/42/events","html_url":"https://github.com/farsightsec/fstrm/issues/42","id":309881858,"number":42,"title":"Support for size-based output file rotation","user":{"login":"kempniu","id":1301146,"avatar_url":"https://avatars0.githubusercontent.com/u/1301146?v=4","gravatar_id":"","url":"https://api.github.com/users/kempniu","html_url":"https://github.com/kempniu","followers_url":"https://api.github.com/users/kempniu/followers","following_url":"https://api.github.com/users/kempniu/following{/other_user}","gists_url":"https://api.github.com/users/kempniu/gists{/gist_id}","starred_url":"https://api.github.com/users/kempniu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kempniu/subscriptions","organizations_url":"https://api.github.com/users/kempniu/orgs","repos_url":"https://api.github.com/users/kempniu/repos","events_url":"https://api.github.com/users/kempniu/events{/privacy}","received_events_url":"https://api.github.com/users/kempniu/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-03-29T19:50:03Z","updated_at":"2018-03-29T20:59:33Z","closed_at":null,"author_association":"NONE","body":"BIND allows dnstap output files to be rolled when their size exceeds a configured limit.  In order to make that possible, though, a set of inelegant workarounds are required in fstrm-related code.  The only way to support such a feature in a clean manner seems to be to get it implemented in fstrm itself.  Such an addition could potentially also be welcomed by other fstrm users.\r\n\r\nThis issue is not really a feature request, but rather a question: would support for size-based output file rolling be considered a welcome addition to fstrm's source code?  I am aware that operational usefulness of size-based rolling can be questioned, but apparently there are people interested in using it.  What is your opinion?"},"comment":{"url":"https://api.github.com/repos/farsightsec/fstrm/issues/comments/377368294","html_url":"https://github.com/farsightsec/fstrm/issues/42#issuecomment-377368294","issue_url":"https://api.github.com/repos/farsightsec/fstrm/issues/42","id":377368294,"user":{"login":"kempniu","id":1301146,"avatar_url":"https://avatars0.githubusercontent.com/u/1301146?v=4","gravatar_id":"","url":"https://api.github.com/users/kempniu","html_url":"https://github.com/kempniu","followers_url":"https://api.github.com/users/kempniu/followers","following_url":"https://api.github.com/users/kempniu/following{/other_user}","gists_url":"https://api.github.com/users/kempniu/gists{/gist_id}","starred_url":"https://api.github.com/users/kempniu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kempniu/subscriptions","organizations_url":"https://api.github.com/users/kempniu/orgs","repos_url":"https://api.github.com/users/kempniu/repos","events_url":"https://api.github.com/users/kempniu/events{/privacy}","received_events_url":"https://api.github.com/users/kempniu/received_events","type":"User","site_admin":false},"created_at":"2018-03-29T20:51:29Z","updated_at":"2018-03-29T20:51:29Z","author_association":"NONE","body":"The following two functions come to mind specifically:\r\n\r\n* [`dns_dt_reopen()`](https://gitlab.isc.org/isc-projects/bind9/blob/52909b5dbeadf13ecbf43e8ee573af23eebd6859/lib/dns/dnstap.c#L333-448)\r\n* [`dt_queue()`](https://gitlab.isc.org/isc-projects/bind9/blob/52909b5dbeadf13ecbf43e8ee573af23eebd6859/lib/dns/dnstap.c#L489-532)\r\n\r\nWhenever it is determined that the dnstap output file is to be rolled, `named` basically stops doing *everything* else, rolls the output file itself, creates a new fstrm writer thread and increments a static variable which is used by each producer thread (i.e. BIND worker threads) to determine whether the writer thread has been closed since the last time this specific producer thread queued an item to a queue; if so, a new queue is retrieved from the writer thread and subsequently used by the producer thread.\r\n\r\nMeanwhile, IIUC, if fstrm supported size-based file rolling \"natively\", BIND worker threads could just shove data into a queue and let fstrm handle output file rolling.  A lot of the complexity could then be removed from BIND, while (probably) not nearly as much code would need to be added to fstrm.  But given the doubts surrounding size-based rolling itself, the question is whether more work should be put into supporting it properly (dropping support for size-based rolling altogether is also being considered).  Hence me soliciting your opinion :)"}}
{"actor":{"display_login":"kempniu"}}