{"action":"created","issue":{"url":"https://api.github.com/repos/3dem/relion/issues/427","repository_url":"https://api.github.com/repos/3dem/relion","labels_url":"https://api.github.com/repos/3dem/relion/issues/427/labels{/name}","comments_url":"https://api.github.com/repos/3dem/relion/issues/427/comments","events_url":"https://api.github.com/repos/3dem/relion/issues/427/events","html_url":"https://github.com/3dem/relion/issues/427","id":395286891,"node_id":"MDU6SXNzdWUzOTUyODY4OTE=","number":427,"title":"Convergence issues with Refine3D","user":{"login":"fbonnet08","id":9093476,"node_id":"MDQ6VXNlcjkwOTM0NzY=","avatar_url":"https://avatars1.githubusercontent.com/u/9093476?v=4","gravatar_id":"","url":"https://api.github.com/users/fbonnet08","html_url":"https://github.com/fbonnet08","followers_url":"https://api.github.com/users/fbonnet08/followers","following_url":"https://api.github.com/users/fbonnet08/following{/other_user}","gists_url":"https://api.github.com/users/fbonnet08/gists{/gist_id}","starred_url":"https://api.github.com/users/fbonnet08/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fbonnet08/subscriptions","organizations_url":"https://api.github.com/users/fbonnet08/orgs","repos_url":"https://api.github.com/users/fbonnet08/repos","events_url":"https://api.github.com/users/fbonnet08/events{/privacy}","received_events_url":"https://api.github.com/users/fbonnet08/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2019-01-02T15:39:30Z","updated_at":"2019-01-02T15:42:57Z","closed_at":null,"author_association":"NONE","body":"Hi,\r\n\r\nthe Refine3D relion_refine_mpi program keeps crashing before convergence. In most of the maps that \r\nI am trying to reconstructs it fails after 10-20 iterations. The maps are small and should fit in memory without any problems.\r\n\r\nThis is on 100k particle sets using 4xGTX-1080Ti Intel® Core™ i7-6900K CPU @ 3.20GHz × 16 with 64GB of RAM. I launch the code with 5 MPIprocs and 4 threads. The size of the particles image width(height) varies from 192-360 pixels\r\n\r\nI keep getting the following message which seems to be memory related:\r\n\r\nexpectation step.\r\nWARNING: Ignoring required free GPU memory amount of 200 MB, due to space insufficiency.\r\nWARNING: The available space on the GPU after initialization (13 MB) might be insufficient for the expectation step.\r\nWARNING: Ignoring required free GPU memory amount of 200 MB, due to space insufficiency.\r\nWARNING: The available space on the GPU after initialization (6 MB) might be insufficient for the expectation step.\r\n Expectation iteration 1\r\n000/??? sec ~~(,_,\">                                                          [oo]WARNING: Ignoring required free GPU memory amount of 200 MB, due to space insufficiency.\r\nWARNING: The available space on the GPU after initialization (6 MB) might be insufficient for the expectation step.\r\n--------------------------------------------------------------------------\r\nMPI_ABORT was invoked on rank 1 in communicator MPI_COMM_WORLD\r\nwith errorcode 1.\r\n\r\nNOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.\r\nYou may or may not see output from other processes, depending on\r\nexactly when Open MPI kills them.\r\n--------------------------------------------------------------------------\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/3dem/relion/issues/comments/450897215","html_url":"https://github.com/3dem/relion/issues/427#issuecomment-450897215","issue_url":"https://api.github.com/repos/3dem/relion/issues/427","id":450897215,"node_id":"MDEyOklzc3VlQ29tbWVudDQ1MDg5NzIxNQ==","user":{"login":"biochem-fan","id":1611483,"node_id":"MDQ6VXNlcjE2MTE0ODM=","avatar_url":"https://avatars3.githubusercontent.com/u/1611483?v=4","gravatar_id":"","url":"https://api.github.com/users/biochem-fan","html_url":"https://github.com/biochem-fan","followers_url":"https://api.github.com/users/biochem-fan/followers","following_url":"https://api.github.com/users/biochem-fan/following{/other_user}","gists_url":"https://api.github.com/users/biochem-fan/gists{/gist_id}","starred_url":"https://api.github.com/users/biochem-fan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/biochem-fan/subscriptions","organizations_url":"https://api.github.com/users/biochem-fan/orgs","repos_url":"https://api.github.com/users/biochem-fan/repos","events_url":"https://api.github.com/users/biochem-fan/events{/privacy}","received_events_url":"https://api.github.com/users/biochem-fan/received_events","type":"User","site_admin":false},"created_at":"2019-01-02T15:42:57Z","updated_at":"2019-01-02T15:42:57Z","author_association":"CONTRIBUTOR","body":"Did you make sure other programs are not occupying the GPU memory? Please check with `nvidia-smi`."}}
{"actor":{"display_login":"biochem-fan"}}