{"action":"created","issue":{"url":"https://api.github.com/repos/HiSPARC/sapphire/issues/72","repository_url":"https://api.github.com/repos/HiSPARC/sapphire","labels_url":"https://api.github.com/repos/HiSPARC/sapphire/issues/72/labels{/name}","comments_url":"https://api.github.com/repos/HiSPARC/sapphire/issues/72/comments","events_url":"https://api.github.com/repos/HiSPARC/sapphire/issues/72/events","html_url":"https://github.com/HiSPARC/sapphire/issues/72","id":90726912,"number":72,"title":"Sorting CORSIKA showers","user":{"login":"153957","id":1433682,"avatar_url":"https://avatars.githubusercontent.com/u/1433682?v=3","gravatar_id":"","url":"https://api.github.com/users/153957","html_url":"https://github.com/153957","followers_url":"https://api.github.com/users/153957/followers","following_url":"https://api.github.com/users/153957/following{/other_user}","gists_url":"https://api.github.com/users/153957/gists{/gist_id}","starred_url":"https://api.github.com/users/153957/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/153957/subscriptions","organizations_url":"https://api.github.com/users/153957/orgs","repos_url":"https://api.github.com/users/153957/repos","events_url":"https://api.github.com/users/153957/events{/privacy}","received_events_url":"https://api.github.com/users/153957/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":{"login":"153957","id":1433682,"avatar_url":"https://avatars.githubusercontent.com/u/1433682?v=3","gravatar_id":"","url":"https://api.github.com/users/153957","html_url":"https://github.com/153957","followers_url":"https://api.github.com/users/153957/followers","following_url":"https://api.github.com/users/153957/following{/other_user}","gists_url":"https://api.github.com/users/153957/gists{/gist_id}","starred_url":"https://api.github.com/users/153957/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/153957/subscriptions","organizations_url":"https://api.github.com/users/153957/orgs","repos_url":"https://api.github.com/users/153957/repos","events_url":"https://api.github.com/users/153957/events{/privacy}","received_events_url":"https://api.github.com/users/153957/received_events","type":"User","site_admin":false},"milestone":null,"comments":42,"created_at":"2015-06-24T16:32:36Z","updated_at":"2016-04-18T14:27:13Z","closed_at":"2016-04-18T14:27:12Z","body":"With the merging of #70 I started a cronjob that creates jobs on Stoomboot to sort the simulations.\r\nIm not certain if it is very smart to run so many jobs at the same time, it may be IO limited, not CPU limited. However, the storage server should be quite fast (40 TB RAID 6, I think)\r\n\r\nThe jobs are currently run on the `short` queue of Stoomboot, here jobs have a limit of 4 hours of walltime. I started the jobs about 3-4 hours ago. some of the larger showers (5 GB `corsika.h5` file) are still running (been running for 3 hours). The output (`corsika_sorted.h5`) is already 4 GB, so the job should be near completion. This does mean that showers larger than 5 GB might not make it within the 4 hours.\r\n\r\nThose jobs that do not make it should fail and not create a `tmp_sorted_flag` (a file to mark the shower as sorted), because of this they are not removed from the `sort_queued.log`, so no new job will be started for them. This is good. We can wait (several days) until an attempt has been made to convert all showers, then wait until all jobs finish. Whatever remains in `sort_queued.log` will need to be resubmitted on the `generic` queue (change submit command in the script)"},"comment":{"url":"https://api.github.com/repos/HiSPARC/sapphire/issues/comments/211403551","html_url":"https://github.com/HiSPARC/sapphire/issues/72#issuecomment-211403551","issue_url":"https://api.github.com/repos/HiSPARC/sapphire/issues/72","id":211403551,"user":{"login":"tomkooij","id":6316468,"avatar_url":"https://avatars.githubusercontent.com/u/6316468?v=3","gravatar_id":"","url":"https://api.github.com/users/tomkooij","html_url":"https://github.com/tomkooij","followers_url":"https://api.github.com/users/tomkooij/followers","following_url":"https://api.github.com/users/tomkooij/following{/other_user}","gists_url":"https://api.github.com/users/tomkooij/gists{/gist_id}","starred_url":"https://api.github.com/users/tomkooij/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tomkooij/subscriptions","organizations_url":"https://api.github.com/users/tomkooij/orgs","repos_url":"https://api.github.com/users/tomkooij/repos","events_url":"https://api.github.com/users/tomkooij/events{/privacy}","received_events_url":"https://api.github.com/users/tomkooij/received_events","type":"User","site_admin":false},"created_at":"2016-04-18T14:27:12Z","updated_at":"2016-04-18T14:27:12Z","body":"Fixed in #118"}}