{"action":"created","issue":{"url":"https://api.github.com/repos/MISP/MISP/issues/1407","repository_url":"https://api.github.com/repos/MISP/MISP","labels_url":"https://api.github.com/repos/MISP/MISP/issues/1407/labels{/name}","comments_url":"https://api.github.com/repos/MISP/MISP/issues/1407/comments","events_url":"https://api.github.com/repos/MISP/MISP/issues/1407/events","html_url":"https://github.com/MISP/MISP/issues/1407","id":169322004,"number":1407,"title":"Proposal add-delete cycles cause sync delay and massive logs","user":{"login":"RichieB2B","id":1461970,"avatar_url":"https://avatars.githubusercontent.com/u/1461970?v=3","gravatar_id":"","url":"https://api.github.com/users/RichieB2B","html_url":"https://github.com/RichieB2B","followers_url":"https://api.github.com/users/RichieB2B/followers","following_url":"https://api.github.com/users/RichieB2B/following{/other_user}","gists_url":"https://api.github.com/users/RichieB2B/gists{/gist_id}","starred_url":"https://api.github.com/users/RichieB2B/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RichieB2B/subscriptions","organizations_url":"https://api.github.com/users/RichieB2B/orgs","repos_url":"https://api.github.com/users/RichieB2B/repos","events_url":"https://api.github.com/users/RichieB2B/events{/privacy}","received_events_url":"https://api.github.com/users/RichieB2B/received_events","type":"User","site_admin":false},"labels":[{"id":209017130,"url":"https://api.github.com/repos/MISP/MISP/labels/potential-bug","name":"potential-bug","color":"eb6420","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-08-04T08:44:31Z","updated_at":"2016-12-12T11:43:02Z","closed_at":null,"body":"### Work environment\n\n| Questions | Answers |\n| --- | --- |\n| Type of issue | Bug, support |\n| OS version (server) | CentOS |\n| OS version (client) | Seven |\n| PHP version | 5.6 |\n| MISP version / git hash | 2.4.49 |\n| Browser | not applicable |\n### Actual behavior\n\nWhen I add two sync servers misp.circl.lu and misppriv.circl.lu syncing becomes very slow.\nSyncing with misp.circl.lu adds 10k proposals which takes 100+ minutes. After this syncing with misppriv.circl.lu removes all these 10k proposals again. This also quickly fils the logs MySQL table and eventually the file system.\n### Expected behavior\n\nSyncing with both misp.circl.lu and misppriv.circl.lu should result in the combined contents of these two MISP instances, not a ping-pong between what is in one and then the other.\n### Steps to reproduce the behavior\n\nSimply add both misp.circl.lu and misppriv.circl.lu as pull sync servers and enable the pull_all scheduled task.\n### Logs, screenshots, configuration dump, ...\n\nAvailable upon request.\n"},"comment":{"url":"https://api.github.com/repos/MISP/MISP/issues/comments/266408000","html_url":"https://github.com/MISP/MISP/issues/1407#issuecomment-266408000","issue_url":"https://api.github.com/repos/MISP/MISP/issues/1407","id":266408000,"user":{"login":"RichieB2B","id":1461970,"avatar_url":"https://avatars.githubusercontent.com/u/1461970?v=3","gravatar_id":"","url":"https://api.github.com/users/RichieB2B","html_url":"https://github.com/RichieB2B","followers_url":"https://api.github.com/users/RichieB2B/followers","following_url":"https://api.github.com/users/RichieB2B/following{/other_user}","gists_url":"https://api.github.com/users/RichieB2B/gists{/gist_id}","starred_url":"https://api.github.com/users/RichieB2B/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RichieB2B/subscriptions","organizations_url":"https://api.github.com/users/RichieB2B/orgs","repos_url":"https://api.github.com/users/RichieB2B/repos","events_url":"https://api.github.com/users/RichieB2B/events{/privacy}","received_events_url":"https://api.github.com/users/RichieB2B/received_events","type":"User","site_admin":false},"created_at":"2016-12-12T11:43:02Z","updated_at":"2016-12-12T11:43:02Z","body":"In 4 months this issue resulted in about 1M delete + 1M add log entries. Can I safely purge those and leave the other logs?\r\n```\r\nmysql> select model, action, count(*) as count from logs group by model,action order by count desc limit 10;\r\n+-----------------+---------+--------+\r\n| model           | action  | count  |\r\n+-----------------+---------+--------+\r\n| ShadowAttribute | delete  | 974483 |\r\n| ShadowAttribute | add     | 973070 |\r\n| User            | auth    |  87677 |\r\n| Attribute       | add     |  67497 |\r\n| Server          | pull    |  16568 |\r\n| Attribute       | edit    |   6688 |\r\n| Event           | edit    |   5753 |\r\n| User            | email   |   5463 |\r\n| Server          | warning |   2614 |\r\n| Attribute       | delete  |   2256 |\r\n+-----------------+---------+--------+\r\n10 rows in set (4.20 sec)\r\n````"}}
{"actor":{"display_login":"RichieB2B"}}