{"action":"closed","number":3804,"pull_request":{"url":"https://api.github.com/repos/gem/oq-engine/pulls/3804","id":196725981,"node_id":"MDExOlB1bGxSZXF1ZXN0MTk2NzI1OTgx","html_url":"https://github.com/gem/oq-engine/pull/3804","diff_url":"https://github.com/gem/oq-engine/pull/3804.diff","patch_url":"https://github.com/gem/oq-engine/pull/3804.patch","issue_url":"https://api.github.com/repos/gem/oq-engine/issues/3804","number":3804,"state":"closed","locked":false,"title":"Removed the event based rupture calculators","user":{"login":"micheles","id":2463856,"node_id":"MDQ6VXNlcjI0NjM4NTY=","avatar_url":"https://avatars1.githubusercontent.com/u/2463856?v=4","gravatar_id":"","url":"https://api.github.com/users/micheles","html_url":"https://github.com/micheles","followers_url":"https://api.github.com/users/micheles/followers","following_url":"https://api.github.com/users/micheles/following{/other_user}","gists_url":"https://api.github.com/users/micheles/gists{/gist_id}","starred_url":"https://api.github.com/users/micheles/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/micheles/subscriptions","organizations_url":"https://api.github.com/users/micheles/orgs","repos_url":"https://api.github.com/users/micheles/repos","events_url":"https://api.github.com/users/micheles/events{/privacy}","received_events_url":"https://api.github.com/users/micheles/received_events","type":"User","site_admin":false},"body":"The change in the prefiltering mechanism introduced in engine 3.1 has far reaching consequences: in particular now the effective logic tree is available even *before* starting the real computation. It means that the event_based_rupture calculator (which was essential because it was determining the effective logic tree) can be removed and the ruptures can be computed as part of the hazard calculation producing GMFs and hazard curves. This saves a lot of memory and of data transfer, since there is no need to send back and forth the ruptures. Moreover the logic of the event based calculation can be substantially simplified (no need of precalculators) and we can reduced the total number of event based calculators from 6 to 4. A big improvement. Extra care has to be taken to avoid the issue of slow tasks.\r\n\r\nWith the change in this PR I was able, for the first time ever, to run the full Guatemala calculation producing 300+ GB of GMFs. Still, I am adding here a check to forbid calculations producing more\r\nthan 2**32 rows in the gmf_data/data table, thus making the Guatemala calculation impossible again ;-)\r\nThe reason is that given the technological limits (speed of reading the disks, speed of NFS, etc) such calculations are terribly inefficient even when barely possible. It is best to raise an error and make the scientists think.\r\n\r\nThis is a first step towards https://github.com/gem/oq-engine/issues/3806.","created_at":"2018-06-22T12:35:14Z","updated_at":"2018-06-25T13:31:41Z","closed_at":"2018-06-25T13:31:41Z","merged_at":"2018-06-25T13:31:41Z","merge_commit_sha":"d3871f50b02c7554f2bb8feabe96dc31f58b2ff0","assignee":null,"assignees":[],"requested_reviewers":[{"login":"raoanirudh","id":1707414,"node_id":"MDQ6VXNlcjE3MDc0MTQ=","avatar_url":"https://avatars1.githubusercontent.com/u/1707414?v=4","gravatar_id":"","url":"https://api.github.com/users/raoanirudh","html_url":"https://github.com/raoanirudh","followers_url":"https://api.github.com/users/raoanirudh/followers","following_url":"https://api.github.com/users/raoanirudh/following{/other_user}","gists_url":"https://api.github.com/users/raoanirudh/gists{/gist_id}","starred_url":"https://api.github.com/users/raoanirudh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/raoanirudh/subscriptions","organizations_url":"https://api.github.com/users/raoanirudh/orgs","repos_url":"https://api.github.com/users/raoanirudh/repos","events_url":"https://api.github.com/users/raoanirudh/events{/privacy}","received_events_url":"https://api.github.com/users/raoanirudh/received_events","type":"User","site_admin":false}],"requested_teams":[],"labels":[{"id":262430947,"node_id":"MDU6TGFiZWwyNjI0MzA5NDc=","url":"https://api.github.com/repos/gem/oq-engine/labels/enhancement","name":"enhancement","color":"009800","default":true},{"id":114147,"node_id":"MDU6TGFiZWwxMTQxNDc=","url":"https://api.github.com/repos/gem/oq-engine/labels/performance","name":"performance","color":"ededed","default":false}],"milestone":{"url":"https://api.github.com/repos/gem/oq-engine/milestones/21","html_url":"https://github.com/gem/oq-engine/milestone/21","labels_url":"https://api.github.com/repos/gem/oq-engine/milestones/21/labels","id":3387469,"node_id":"MDk6TWlsZXN0b25lMzM4NzQ2OQ==","number":21,"title":"Engine 3.2.0","description":null,"creator":{"login":"micheles","id":2463856,"node_id":"MDQ6VXNlcjI0NjM4NTY=","avatar_url":"https://avatars1.githubusercontent.com/u/2463856?v=4","gravatar_id":"","url":"https://api.github.com/users/micheles","html_url":"https://github.com/micheles","followers_url":"https://api.github.com/users/micheles/followers","following_url":"https://api.github.com/users/micheles/following{/other_user}","gists_url":"https://api.github.com/users/micheles/gists{/gist_id}","starred_url":"https://api.github.com/users/micheles/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/micheles/subscriptions","organizations_url":"https://api.github.com/users/micheles/orgs","repos_url":"https://api.github.com/users/micheles/repos","events_url":"https://api.github.com/users/micheles/events{/privacy}","received_events_url":"https://api.github.com/users/micheles/received_events","type":"User","site_admin":false},"open_issues":13,"closed_issues":53,"state":"open","created_at":"2018-05-30T13:42:00Z","updated_at":"2018-06-25T13:31:41Z","due_on":null,"closed_at":null},"commits_url":"https://api.github.com/repos/gem/oq-engine/pulls/3804/commits","review_comments_url":"https://api.github.com/repos/gem/oq-engine/pulls/3804/comments","review_comment_url":"https://api.github.com/repos/gem/oq-engine/pulls/comments{/number}","comments_url":"https://api.github.com/repos/gem/oq-engine/issues/3804/comments","statuses_url":"https://api.github.com/repos/gem/oq-engine/statuses/a2083d44e429fc65e8c30d309de4dab464284b9c","head":{"label":"gem:event_based","ref":"event_based","sha":"a2083d44e429fc65e8c30d309de4dab464284b9c","user":{"login":"gem","id":316546,"node_id":"MDEyOk9yZ2FuaXphdGlvbjMxNjU0Ng==","avatar_url":"https://avatars1.githubusercontent.com/u/316546?v=4","gravatar_id":"","url":"https://api.github.com/users/gem","html_url":"https://github.com/gem","followers_url":"https://api.github.com/users/gem/followers","following_url":"https://api.github.com/users/gem/following{/other_user}","gists_url":"https://api.github.com/users/gem/gists{/gist_id}","starred_url":"https://api.github.com/users/gem/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gem/subscriptions","organizations_url":"https://api.github.com/users/gem/orgs","repos_url":"https://api.github.com/users/gem/repos","events_url":"https://api.github.com/users/gem/events{/privacy}","received_events_url":"https://api.github.com/users/gem/received_events","type":"Organization","site_admin":false},"repo":{"id":744916,"node_id":"MDEwOlJlcG9zaXRvcnk3NDQ5MTY=","name":"oq-engine","full_name":"gem/oq-engine","owner":{"login":"gem","id":316546,"node_id":"MDEyOk9yZ2FuaXphdGlvbjMxNjU0Ng==","avatar_url":"https://avatars1.githubusercontent.com/u/316546?v=4","gravatar_id":"","url":"https://api.github.com/users/gem","html_url":"https://github.com/gem","followers_url":"https://api.github.com/users/gem/followers","following_url":"https://api.github.com/users/gem/following{/other_user}","gists_url":"https://api.github.com/users/gem/gists{/gist_id}","starred_url":"https://api.github.com/users/gem/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gem/subscriptions","organizations_url":"https://api.github.com/users/gem/orgs","repos_url":"https://api.github.com/users/gem/repos","events_url":"https://api.github.com/users/gem/events{/privacy}","received_events_url":"https://api.github.com/users/gem/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/gem/oq-engine","description":"OpenQuake's Engine for Seismic Hazard and Risk Analysis","fork":false,"url":"https://api.github.com/repos/gem/oq-engine","forks_url":"https://api.github.com/repos/gem/oq-engine/forks","keys_url":"https://api.github.com/repos/gem/oq-engine/keys{/key_id}","collaborators_url":"https://api.github.com/repos/gem/oq-engine/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/gem/oq-engine/teams","hooks_url":"https://api.github.com/repos/gem/oq-engine/hooks","issue_events_url":"https://api.github.com/repos/gem/oq-engine/issues/events{/number}","events_url":"https://api.github.com/repos/gem/oq-engine/events","assignees_url":"https://api.github.com/repos/gem/oq-engine/assignees{/user}","branches_url":"https://api.github.com/repos/gem/oq-engine/branches{/branch}","tags_url":"https://api.github.com/repos/gem/oq-engine/tags","blobs_url":"https://api.github.com/repos/gem/oq-engine/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/gem/oq-engine/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/gem/oq-engine/git/refs{/sha}","trees_url":"https://api.github.com/repos/gem/oq-engine/git/trees{/sha}","statuses_url":"https://api.github.com/repos/gem/oq-engine/statuses/{sha}","languages_url":"https://api.github.com/repos/gem/oq-engine/languages","stargazers_url":"https://api.github.com/repos/gem/oq-engine/stargazers","contributors_url":"https://api.github.com/repos/gem/oq-engine/contributors","subscribers_url":"https://api.github.com/repos/gem/oq-engine/subscribers","subscription_url":"https://api.github.com/repos/gem/oq-engine/subscription","commits_url":"https://api.github.com/repos/gem/oq-engine/commits{/sha}","git_commits_url":"https://api.github.com/repos/gem/oq-engine/git/commits{/sha}","comments_url":"https://api.github.com/repos/gem/oq-engine/comments{/number}","issue_comment_url":"https://api.github.com/repos/gem/oq-engine/issues/comments{/number}","contents_url":"https://api.github.com/repos/gem/oq-engine/contents/{+path}","compare_url":"https://api.github.com/repos/gem/oq-engine/compare/{base}...{head}","merges_url":"https://api.github.com/repos/gem/oq-engine/merges","archive_url":"https://api.github.com/repos/gem/oq-engine/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/gem/oq-engine/downloads","issues_url":"https://api.github.com/repos/gem/oq-engine/issues{/number}","pulls_url":"https://api.github.com/repos/gem/oq-engine/pulls{/number}","milestones_url":"https://api.github.com/repos/gem/oq-engine/milestones{/number}","notifications_url":"https://api.github.com/repos/gem/oq-engine/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/gem/oq-engine/labels{/name}","releases_url":"https://api.github.com/repos/gem/oq-engine/releases{/id}","deployments_url":"https://api.github.com/repos/gem/oq-engine/deployments","created_at":"2010-06-28T14:59:42Z","updated_at":"2018-06-25T13:08:13Z","pushed_at":"2018-06-25T13:31:41Z","git_url":"git://github.com/gem/oq-engine.git","ssh_url":"git@github.com:gem/oq-engine.git","clone_url":"https://github.com/gem/oq-engine.git","svn_url":"https://github.com/gem/oq-engine","homepage":"https://github.com/gem/oq-engine/#openquake-engine","size":643571,"stargazers_count":116,"watchers_count":116,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":93,"mirror_url":null,"archived":false,"open_issues_count":40,"license":{"key":"agpl-3.0","name":"GNU Affero General Public License v3.0","spdx_id":"AGPL-3.0","url":"https://api.github.com/licenses/agpl-3.0","node_id":"MDc6TGljZW5zZTE="},"forks":93,"open_issues":40,"watchers":116,"default_branch":"master"}},"base":{"label":"gem:master","ref":"master","sha":"928625df90542d17ec40ae6eb08f75e4506c5150","user":{"login":"gem","id":316546,"node_id":"MDEyOk9yZ2FuaXphdGlvbjMxNjU0Ng==","avatar_url":"https://avatars1.githubusercontent.com/u/316546?v=4","gravatar_id":"","url":"https://api.github.com/users/gem","html_url":"https://github.com/gem","followers_url":"https://api.github.com/users/gem/followers","following_url":"https://api.github.com/users/gem/following{/other_user}","gists_url":"https://api.github.com/users/gem/gists{/gist_id}","starred_url":"https://api.github.com/users/gem/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gem/subscriptions","organizations_url":"https://api.github.com/users/gem/orgs","repos_url":"https://api.github.com/users/gem/repos","events_url":"https://api.github.com/users/gem/events{/privacy}","received_events_url":"https://api.github.com/users/gem/received_events","type":"Organization","site_admin":false},"repo":{"id":744916,"node_id":"MDEwOlJlcG9zaXRvcnk3NDQ5MTY=","name":"oq-engine","full_name":"gem/oq-engine","owner":{"login":"gem","id":316546,"node_id":"MDEyOk9yZ2FuaXphdGlvbjMxNjU0Ng==","avatar_url":"https://avatars1.githubusercontent.com/u/316546?v=4","gravatar_id":"","url":"https://api.github.com/users/gem","html_url":"https://github.com/gem","followers_url":"https://api.github.com/users/gem/followers","following_url":"https://api.github.com/users/gem/following{/other_user}","gists_url":"https://api.github.com/users/gem/gists{/gist_id}","starred_url":"https://api.github.com/users/gem/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gem/subscriptions","organizations_url":"https://api.github.com/users/gem/orgs","repos_url":"https://api.github.com/users/gem/repos","events_url":"https://api.github.com/users/gem/events{/privacy}","received_events_url":"https://api.github.com/users/gem/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/gem/oq-engine","description":"OpenQuake's Engine for Seismic Hazard and Risk Analysis","fork":false,"url":"https://api.github.com/repos/gem/oq-engine","forks_url":"https://api.github.com/repos/gem/oq-engine/forks","keys_url":"https://api.github.com/repos/gem/oq-engine/keys{/key_id}","collaborators_url":"https://api.github.com/repos/gem/oq-engine/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/gem/oq-engine/teams","hooks_url":"https://api.github.com/repos/gem/oq-engine/hooks","issue_events_url":"https://api.github.com/repos/gem/oq-engine/issues/events{/number}","events_url":"https://api.github.com/repos/gem/oq-engine/events","assignees_url":"https://api.github.com/repos/gem/oq-engine/assignees{/user}","branches_url":"https://api.github.com/repos/gem/oq-engine/branches{/branch}","tags_url":"https://api.github.com/repos/gem/oq-engine/tags","blobs_url":"https://api.github.com/repos/gem/oq-engine/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/gem/oq-engine/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/gem/oq-engine/git/refs{/sha}","trees_url":"https://api.github.com/repos/gem/oq-engine/git/trees{/sha}","statuses_url":"https://api.github.com/repos/gem/oq-engine/statuses/{sha}","languages_url":"https://api.github.com/repos/gem/oq-engine/languages","stargazers_url":"https://api.github.com/repos/gem/oq-engine/stargazers","contributors_url":"https://api.github.com/repos/gem/oq-engine/contributors","subscribers_url":"https://api.github.com/repos/gem/oq-engine/subscribers","subscription_url":"https://api.github.com/repos/gem/oq-engine/subscription","commits_url":"https://api.github.com/repos/gem/oq-engine/commits{/sha}","git_commits_url":"https://api.github.com/repos/gem/oq-engine/git/commits{/sha}","comments_url":"https://api.github.com/repos/gem/oq-engine/comments{/number}","issue_comment_url":"https://api.github.com/repos/gem/oq-engine/issues/comments{/number}","contents_url":"https://api.github.com/repos/gem/oq-engine/contents/{+path}","compare_url":"https://api.github.com/repos/gem/oq-engine/compare/{base}...{head}","merges_url":"https://api.github.com/repos/gem/oq-engine/merges","archive_url":"https://api.github.com/repos/gem/oq-engine/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/gem/oq-engine/downloads","issues_url":"https://api.github.com/repos/gem/oq-engine/issues{/number}","pulls_url":"https://api.github.com/repos/gem/oq-engine/pulls{/number}","milestones_url":"https://api.github.com/repos/gem/oq-engine/milestones{/number}","notifications_url":"https://api.github.com/repos/gem/oq-engine/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/gem/oq-engine/labels{/name}","releases_url":"https://api.github.com/repos/gem/oq-engine/releases{/id}","deployments_url":"https://api.github.com/repos/gem/oq-engine/deployments","created_at":"2010-06-28T14:59:42Z","updated_at":"2018-06-25T13:08:13Z","pushed_at":"2018-06-25T13:31:41Z","git_url":"git://github.com/gem/oq-engine.git","ssh_url":"git@github.com:gem/oq-engine.git","clone_url":"https://github.com/gem/oq-engine.git","svn_url":"https://github.com/gem/oq-engine","homepage":"https://github.com/gem/oq-engine/#openquake-engine","size":643571,"stargazers_count":116,"watchers_count":116,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":93,"mirror_url":null,"archived":false,"open_issues_count":40,"license":{"key":"agpl-3.0","name":"GNU Affero General Public License v3.0","spdx_id":"AGPL-3.0","url":"https://api.github.com/licenses/agpl-3.0","node_id":"MDc6TGljZW5zZTE="},"forks":93,"open_issues":40,"watchers":116,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/gem/oq-engine/pulls/3804"},"html":{"href":"https://github.com/gem/oq-engine/pull/3804"},"issue":{"href":"https://api.github.com/repos/gem/oq-engine/issues/3804"},"comments":{"href":"https://api.github.com/repos/gem/oq-engine/issues/3804/comments"},"review_comments":{"href":"https://api.github.com/repos/gem/oq-engine/pulls/3804/comments"},"review_comment":{"href":"https://api.github.com/repos/gem/oq-engine/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/gem/oq-engine/pulls/3804/commits"},"statuses":{"href":"https://api.github.com/repos/gem/oq-engine/statuses/a2083d44e429fc65e8c30d309de4dab464284b9c"}},"author_association":"CONTRIBUTOR","merged":true,"mergeable":null,"rebaseable":null,"mergeable_state":"unknown","merged_by":{"login":"micheles","id":2463856,"node_id":"MDQ6VXNlcjI0NjM4NTY=","avatar_url":"https://avatars1.githubusercontent.com/u/2463856?v=4","gravatar_id":"","url":"https://api.github.com/users/micheles","html_url":"https://github.com/micheles","followers_url":"https://api.github.com/users/micheles/followers","following_url":"https://api.github.com/users/micheles/following{/other_user}","gists_url":"https://api.github.com/users/micheles/gists{/gist_id}","starred_url":"https://api.github.com/users/micheles/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/micheles/subscriptions","organizations_url":"https://api.github.com/users/micheles/orgs","repos_url":"https://api.github.com/users/micheles/repos","events_url":"https://api.github.com/users/micheles/events{/privacy}","received_events_url":"https://api.github.com/users/micheles/received_events","type":"User","site_admin":false},"comments":0,"review_comments":0,"maintainer_can_modify":false,"commits":34,"additions":310,"deletions":368,"changed_files":21}}
{"actor":{"display_login":"micheles"}}