{"action":"created","issue":{"url":"https://api.github.com/repos/DigitalPebble/storm-crawler/issues/613","repository_url":"https://api.github.com/repos/DigitalPebble/storm-crawler","labels_url":"https://api.github.com/repos/DigitalPebble/storm-crawler/issues/613/labels{/name}","comments_url":"https://api.github.com/repos/DigitalPebble/storm-crawler/issues/613/comments","events_url":"https://api.github.com/repos/DigitalPebble/storm-crawler/issues/613/events","html_url":"https://github.com/DigitalPebble/storm-crawler/pull/613","id":363967391,"node_id":"MDExOlB1bGxSZXF1ZXN0MjE4MjY5NTk4","number":613,"title":"Batch PreparedStatements in SQL status updater bolt, fixes #610","user":{"login":"jnioche","id":218319,"node_id":"MDQ6VXNlcjIxODMxOQ==","avatar_url":"https://avatars1.githubusercontent.com/u/218319?v=4","gravatar_id":"","url":"https://api.github.com/users/jnioche","html_url":"https://github.com/jnioche","followers_url":"https://api.github.com/users/jnioche/followers","following_url":"https://api.github.com/users/jnioche/following{/other_user}","gists_url":"https://api.github.com/users/jnioche/gists{/gist_id}","starred_url":"https://api.github.com/users/jnioche/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jnioche/subscriptions","organizations_url":"https://api.github.com/users/jnioche/orgs","repos_url":"https://api.github.com/users/jnioche/repos","events_url":"https://api.github.com/users/jnioche/events{/privacy}","received_events_url":"https://api.github.com/users/jnioche/received_events","type":"User","site_admin":false},"labels":[{"id":1067895234,"node_id":"MDU6TGFiZWwxMDY3ODk1MjM0","url":"https://api.github.com/repos/DigitalPebble/storm-crawler/labels/SQL","name":"SQL","color":"ce0c8a","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/DigitalPebble/storm-crawler/milestones/20","html_url":"https://github.com/DigitalPebble/storm-crawler/milestone/20","labels_url":"https://api.github.com/repos/DigitalPebble/storm-crawler/milestones/20/labels","id":3424546,"node_id":"MDk6TWlsZXN0b25lMzQyNDU0Ng==","number":20,"title":"1.11","description":"","creator":{"login":"jnioche","id":218319,"node_id":"MDQ6VXNlcjIxODMxOQ==","avatar_url":"https://avatars1.githubusercontent.com/u/218319?v=4","gravatar_id":"","url":"https://api.github.com/users/jnioche","html_url":"https://github.com/jnioche","followers_url":"https://api.github.com/users/jnioche/followers","following_url":"https://api.github.com/users/jnioche/following{/other_user}","gists_url":"https://api.github.com/users/jnioche/gists{/gist_id}","starred_url":"https://api.github.com/users/jnioche/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jnioche/subscriptions","organizations_url":"https://api.github.com/users/jnioche/orgs","repos_url":"https://api.github.com/users/jnioche/repos","events_url":"https://api.github.com/users/jnioche/events{/privacy}","received_events_url":"https://api.github.com/users/jnioche/received_events","type":"User","site_admin":false},"open_issues":1,"closed_issues":19,"state":"open","created_at":"2018-06-14T10:51:56Z","updated_at":"2018-09-27T10:36:53Z","due_on":"2018-10-01T07:00:00Z","closed_at":null},"comments":7,"created_at":"2018-09-26T10:47:33Z","updated_at":"2018-09-27T12:01:21Z","closed_at":null,"author_association":"MEMBER","pull_request":{"url":"https://api.github.com/repos/DigitalPebble/storm-crawler/pulls/613","html_url":"https://github.com/DigitalPebble/storm-crawler/pull/613","diff_url":"https://github.com/DigitalPebble/storm-crawler/pull/613.diff","patch_url":"https://github.com/DigitalPebble/storm-crawler/pull/613.patch"},"body":"This PR changes the behaviour of the SQL StatusUpdaterBolt by batching the INSERTs, i.e. the discovered URLs which are far more frequent than updates.\r\n \r\n@cruftex plz let me know what you think. we can make it SQL agnostic in a separate step."},"comment":{"url":"https://api.github.com/repos/DigitalPebble/storm-crawler/issues/comments/425064456","html_url":"https://github.com/DigitalPebble/storm-crawler/pull/613#issuecomment-425064456","issue_url":"https://api.github.com/repos/DigitalPebble/storm-crawler/issues/613","id":425064456,"node_id":"MDEyOklzc3VlQ29tbWVudDQyNTA2NDQ1Ng==","user":{"login":"rzo1","id":13417392,"node_id":"MDQ6VXNlcjEzNDE3Mzky","avatar_url":"https://avatars3.githubusercontent.com/u/13417392?v=4","gravatar_id":"","url":"https://api.github.com/users/rzo1","html_url":"https://github.com/rzo1","followers_url":"https://api.github.com/users/rzo1/followers","following_url":"https://api.github.com/users/rzo1/following{/other_user}","gists_url":"https://api.github.com/users/rzo1/gists{/gist_id}","starred_url":"https://api.github.com/users/rzo1/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rzo1/subscriptions","organizations_url":"https://api.github.com/users/rzo1/orgs","repos_url":"https://api.github.com/users/rzo1/repos","events_url":"https://api.github.com/users/rzo1/events{/privacy}","received_events_url":"https://api.github.com/users/rzo1/received_events","type":"User","site_admin":false},"created_at":"2018-09-27T12:01:21Z","updated_at":"2018-09-27T12:01:21Z","author_association":"NONE","body":"I thought about automatic re-connection in case of timeout, stale/idle connections or others things like that - the things a DBA has to deal with it otherwise. \r\n\r\nIt really has some benefits but can easily added later if the implementation does the right things :) - we even use something similar combined with the ES StormCrawler variant for some research purpose.\r\n"}}
{"actor":{"display_login":"rzo1"}}