{"action":"created","issue":{"url":"https://api.github.com/repos/CESNET/ipfixcol/issues/162","repository_url":"https://api.github.com/repos/CESNET/ipfixcol","labels_url":"https://api.github.com/repos/CESNET/ipfixcol/issues/162/labels{/name}","comments_url":"https://api.github.com/repos/CESNET/ipfixcol/issues/162/comments","events_url":"https://api.github.com/repos/CESNET/ipfixcol/issues/162/events","html_url":"https://github.com/CESNET/ipfixcol/issues/162","id":140029093,"number":162,"title":"Question - How to get the ipfix filewriter to rotate","user":{"login":"jepries","id":10326075,"avatar_url":"https://avatars.githubusercontent.com/u/10326075?v=3","gravatar_id":"","url":"https://api.github.com/users/jepries","html_url":"https://github.com/jepries","followers_url":"https://api.github.com/users/jepries/followers","following_url":"https://api.github.com/users/jepries/following{/other_user}","gists_url":"https://api.github.com/users/jepries/gists{/gist_id}","starred_url":"https://api.github.com/users/jepries/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jepries/subscriptions","organizations_url":"https://api.github.com/users/jepries/orgs","repos_url":"https://api.github.com/users/jepries/repos","events_url":"https://api.github.com/users/jepries/events{/privacy}","received_events_url":"https://api.github.com/users/jepries/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-03-10T22:35:38Z","updated_at":"2016-03-13T02:23:02Z","closed_at":null,"body":"I did some capacity tests writing out received packets via JSON and via ipfix file.  The ipfix native format is way better, don't start losing data until > 15K packets per second, whereas I start losing data using json output at 4k packets per second.\r\n\r\nNo worries, I can store to ipfix and post process the ipfix file to json later.  Question is, how do I get it to rotate out the ipfix files (or do I need to make my own plugin to do that)?\r\n\r\nThanks guys, this code is awesome, really making it work for me."},"comment":{"url":"https://api.github.com/repos/CESNET/ipfixcol/issues/comments/195854015","html_url":"https://github.com/CESNET/ipfixcol/issues/162#issuecomment-195854015","issue_url":"https://api.github.com/repos/CESNET/ipfixcol/issues/162","id":195854015,"user":{"login":"jepries","id":10326075,"avatar_url":"https://avatars.githubusercontent.com/u/10326075?v=3","gravatar_id":"","url":"https://api.github.com/users/jepries","html_url":"https://github.com/jepries","followers_url":"https://api.github.com/users/jepries/followers","following_url":"https://api.github.com/users/jepries/following{/other_user}","gists_url":"https://api.github.com/users/jepries/gists{/gist_id}","starred_url":"https://api.github.com/users/jepries/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jepries/subscriptions","organizations_url":"https://api.github.com/users/jepries/orgs","repos_url":"https://api.github.com/users/jepries/repos","events_url":"https://api.github.com/users/jepries/events{/privacy}","received_events_url":"https://api.github.com/users/jepries/received_events","type":"User","site_admin":false},"created_at":"2016-03-13T02:23:02Z","updated_at":"2016-03-13T02:23:02Z","body":"I'll answer my own question :).... I wrote a ipfix output module (shamelessly combined json code and ipfix storage code) called ripfix (for rotate ipfix).  Works.... but... the output ipfix files that come later don't contain the templates, so passing the files through ipfixcol again using ipfix input and json output, json doesn't have the templates yet to parse the data.. so I lose records in json....\r\n\r\nNew question: Is there a way I can seed the new IPFIX file with all known templates before I hand off writing new packets to it?  I looked through template_manager and it wasn't obvious to me.\r\n\r\nNot sure how else to increase the speed, I guess I have to look at fastbit, but then have to convert it to json."}}