{"action":"created","issue":{"url":"https://api.github.com/repos/Illumina/manta/issues/154","repository_url":"https://api.github.com/repos/Illumina/manta","labels_url":"https://api.github.com/repos/Illumina/manta/issues/154/labels{/name}","comments_url":"https://api.github.com/repos/Illumina/manta/issues/154/comments","events_url":"https://api.github.com/repos/Illumina/manta/issues/154/events","html_url":"https://github.com/Illumina/manta/issues/154","id":369597026,"node_id":"MDU6SXNzdWUzNjk1OTcwMjY=","number":154,"title":"Understanding excessive memory usage","user":{"login":"ernfrid","id":397990,"node_id":"MDQ6VXNlcjM5Nzk5MA==","avatar_url":"https://avatars1.githubusercontent.com/u/397990?v=4","gravatar_id":"","url":"https://api.github.com/users/ernfrid","html_url":"https://github.com/ernfrid","followers_url":"https://api.github.com/users/ernfrid/followers","following_url":"https://api.github.com/users/ernfrid/following{/other_user}","gists_url":"https://api.github.com/users/ernfrid/gists{/gist_id}","starred_url":"https://api.github.com/users/ernfrid/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ernfrid/subscriptions","organizations_url":"https://api.github.com/users/ernfrid/orgs","repos_url":"https://api.github.com/users/ernfrid/repos","events_url":"https://api.github.com/users/ernfrid/events{/privacy}","received_events_url":"https://api.github.com/users/ernfrid/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"ctsa","id":1349103,"node_id":"MDQ6VXNlcjEzNDkxMDM=","avatar_url":"https://avatars3.githubusercontent.com/u/1349103?v=4","gravatar_id":"","url":"https://api.github.com/users/ctsa","html_url":"https://github.com/ctsa","followers_url":"https://api.github.com/users/ctsa/followers","following_url":"https://api.github.com/users/ctsa/following{/other_user}","gists_url":"https://api.github.com/users/ctsa/gists{/gist_id}","starred_url":"https://api.github.com/users/ctsa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ctsa/subscriptions","organizations_url":"https://api.github.com/users/ctsa/orgs","repos_url":"https://api.github.com/users/ctsa/repos","events_url":"https://api.github.com/users/ctsa/events{/privacy}","received_events_url":"https://api.github.com/users/ctsa/received_events","type":"User","site_admin":false},"assignees":[{"login":"ctsa","id":1349103,"node_id":"MDQ6VXNlcjEzNDkxMDM=","avatar_url":"https://avatars3.githubusercontent.com/u/1349103?v=4","gravatar_id":"","url":"https://api.github.com/users/ctsa","html_url":"https://github.com/ctsa","followers_url":"https://api.github.com/users/ctsa/followers","following_url":"https://api.github.com/users/ctsa/following{/other_user}","gists_url":"https://api.github.com/users/ctsa/gists{/gist_id}","starred_url":"https://api.github.com/users/ctsa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ctsa/subscriptions","organizations_url":"https://api.github.com/users/ctsa/orgs","repos_url":"https://api.github.com/users/ctsa/repos","events_url":"https://api.github.com/users/ctsa/events{/privacy}","received_events_url":"https://api.github.com/users/ctsa/received_events","type":"User","site_admin":false}],"milestone":null,"comments":18,"created_at":"2018-10-12T15:19:14Z","updated_at":"2018-11-19T23:48:25Z","closed_at":null,"author_association":"NONE","body":"I've been running a large number of ~ 30X human CRAMs (aligned to GRCh38DH) through Manta for germline SV detection. I've typically been providing 16GB of RAM and 8 cores. For the majority of samples, this is sufficient, but for a small subset I need additional RAM and I've recently encountered a sample that utilized ~70GB of RAM to complete.\r\n\r\nUnfortunately, I will be unable to share the CRAM in question. Is there any part of the output or log files that might shed light on the source of this excess usage? In particular, I'm curious if there is a particular region or contig that I might exclude from my analysis to reduce the usage (or use to predict excess usage in advance)."},"comment":{"url":"https://api.github.com/repos/Illumina/manta/issues/comments/440083971","html_url":"https://github.com/Illumina/manta/issues/154#issuecomment-440083971","issue_url":"https://api.github.com/repos/Illumina/manta/issues/154","id":440083971,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0MDA4Mzk3MQ==","user":{"login":"jkbonfield","id":2210525,"node_id":"MDQ6VXNlcjIyMTA1MjU=","avatar_url":"https://avatars0.githubusercontent.com/u/2210525?v=4","gravatar_id":"","url":"https://api.github.com/users/jkbonfield","html_url":"https://github.com/jkbonfield","followers_url":"https://api.github.com/users/jkbonfield/followers","following_url":"https://api.github.com/users/jkbonfield/following{/other_user}","gists_url":"https://api.github.com/users/jkbonfield/gists{/gist_id}","starred_url":"https://api.github.com/users/jkbonfield/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jkbonfield/subscriptions","organizations_url":"https://api.github.com/users/jkbonfield/orgs","repos_url":"https://api.github.com/users/jkbonfield/repos","events_url":"https://api.github.com/users/jkbonfield/events{/privacy}","received_events_url":"https://api.github.com/users/jkbonfield/received_events","type":"User","site_admin":false},"created_at":"2018-11-19T23:48:25Z","updated_at":"2018-11-19T23:48:25Z","author_association":"NONE","body":"> I also tried @jkbonfield's `samtools view -c` test and see nearly identical RAM utilization. I haven't yet tried iterating through many regions.\r\n\r\nDo you mean that samtools view -c is also using 40+GB of RAM?  If so then it's definitely something in our code causing this high memory usage!  Odd and a cause for concern.  It does sound rather like a memory leak.\r\n\r\nIs this tied to using multi-threaded mode?  Ie with no threads (no -@ option) does it also gradually climb in memory?"}}
{"actor":{"display_login":"jkbonfield"}}