{"action":"created","issue":{"url":"https://api.github.com/repos/ntop/ntopng/issues/359","labels_url":"https://api.github.com/repos/ntop/ntopng/issues/359/labels{/name}","comments_url":"https://api.github.com/repos/ntop/ntopng/issues/359/comments","events_url":"https://api.github.com/repos/ntop/ntopng/issues/359/events","html_url":"https://github.com/ntop/ntopng/issues/359","id":126729174,"number":359,"title":"Dump Flows -F with mysql results in high packets drops","user":{"login":"h8h","id":953856,"avatar_url":"https://avatars.githubusercontent.com/u/953856?v=3","gravatar_id":"","url":"https://api.github.com/users/h8h","html_url":"https://github.com/h8h","followers_url":"https://api.github.com/users/h8h/followers","following_url":"https://api.github.com/users/h8h/following{/other_user}","gists_url":"https://api.github.com/users/h8h/gists{/gist_id}","starred_url":"https://api.github.com/users/h8h/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/h8h/subscriptions","organizations_url":"https://api.github.com/users/h8h/orgs","repos_url":"https://api.github.com/users/h8h/repos","events_url":"https://api.github.com/users/h8h/events{/privacy}","received_events_url":"https://api.github.com/users/h8h/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2016-01-14T19:26:38Z","updated_at":"2016-01-23T00:49:34Z","closed_at":null,"body":"I have a jessie kvm instance with ntopng ([latest sourceforge build](http://sourceforge.net/projects/ntop/files/ntopng/ntopng-2.2.tar.gz/download)) running inside. I decide to dump the expired flows to a mysql database. \r\n\r\nIf I activate this option the dropped \tpackets raise up to 99 % (from 0.45%) and the bandwidth drops to nearly 0 - 1 MB/s (from 90MB/s). \r\n\r\nIt seems that it doesn't matter whether the database is running inside the kvm or outside on the host.\r\n\r\nThe database always listens on an other interface which is not monitored by ntopng.\r\n\r\nHas someone shared the same expirence?"},"comment":{"url":"https://api.github.com/repos/ntop/ntopng/issues/comments/174099992","html_url":"https://github.com/ntop/ntopng/issues/359#issuecomment-174099992","issue_url":"https://api.github.com/repos/ntop/ntopng/issues/359","id":174099992,"user":{"login":"simonemainardi","id":3397663,"avatar_url":"https://avatars.githubusercontent.com/u/3397663?v=3","gravatar_id":"","url":"https://api.github.com/users/simonemainardi","html_url":"https://github.com/simonemainardi","followers_url":"https://api.github.com/users/simonemainardi/followers","following_url":"https://api.github.com/users/simonemainardi/following{/other_user}","gists_url":"https://api.github.com/users/simonemainardi/gists{/gist_id}","starred_url":"https://api.github.com/users/simonemainardi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/simonemainardi/subscriptions","organizations_url":"https://api.github.com/users/simonemainardi/orgs","repos_url":"https://api.github.com/users/simonemainardi/repos","events_url":"https://api.github.com/users/simonemainardi/events{/privacy}","received_events_url":"https://api.github.com/users/simonemainardi/received_events","type":"User","site_admin":false},"created_at":"2016-01-23T00:49:34Z","updated_at":"2016-01-23T00:49:34Z","body":"Whereas Elastic Search dump is asynchronous and occurs in a separate thread, MySQL flow export is synchronous and occurs in the same thread that handles the interface traffic. So it is likely that high volumes of exported flows will interfere negatively with the monitoring.\r\n\r\nAs a quick workaround, you may try and tune MySQL configuration to increase threads, memory, and buffer sizes. In this way, flow export could happen at an higher speed.\r\n\r\nAs a reminder for ntopng devs, we should\r\n1. make MySQL dump asynchronous;\r\n2. buffer flows in order to issue a single `INSERT INTO` for hundreds/thousands of flows\r\n"}}