{"action":"created","issue":{"url":"https://api.github.com/repos/vgteam/vg/issues/1737","repository_url":"https://api.github.com/repos/vgteam/vg","labels_url":"https://api.github.com/repos/vgteam/vg/issues/1737/labels{/name}","comments_url":"https://api.github.com/repos/vgteam/vg/issues/1737/comments","events_url":"https://api.github.com/repos/vgteam/vg/issues/1737/events","html_url":"https://github.com/vgteam/vg/issues/1737","id":332137821,"node_id":"MDU6SXNzdWUzMzIxMzc4MjE=","number":1737,"title":"Construction crash introduced with Safe SVs","user":{"login":"adamnovak","id":5062495,"node_id":"MDQ6VXNlcjUwNjI0OTU=","avatar_url":"https://avatars1.githubusercontent.com/u/5062495?v=4","gravatar_id":"","url":"https://api.github.com/users/adamnovak","html_url":"https://github.com/adamnovak","followers_url":"https://api.github.com/users/adamnovak/followers","following_url":"https://api.github.com/users/adamnovak/following{/other_user}","gists_url":"https://api.github.com/users/adamnovak/gists{/gist_id}","starred_url":"https://api.github.com/users/adamnovak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adamnovak/subscriptions","organizations_url":"https://api.github.com/users/adamnovak/orgs","repos_url":"https://api.github.com/users/adamnovak/repos","events_url":"https://api.github.com/users/adamnovak/events{/privacy}","received_events_url":"https://api.github.com/users/adamnovak/received_events","type":"User","site_admin":false},"labels":[{"id":936456159,"node_id":"MDU6TGFiZWw5MzY0NTYxNTk=","url":"https://api.github.com/repos/vgteam/vg/labels/HAMP","name":"HAMP","color":"c2f704","default":false},{"id":134359496,"node_id":"MDU6TGFiZWwxMzQzNTk0OTY=","url":"https://api.github.com/repos/vgteam/vg/labels/bug","name":"bug","color":"fc2929","default":true},{"id":280142072,"node_id":"MDU6TGFiZWwyODAxNDIwNzI=","url":"https://api.github.com/repos/vgteam/vg/labels/in%20progress","name":"in progress","color":"ededed","default":false}],"state":"open","locked":false,"assignee":{"login":"adamnovak","id":5062495,"node_id":"MDQ6VXNlcjUwNjI0OTU=","avatar_url":"https://avatars1.githubusercontent.com/u/5062495?v=4","gravatar_id":"","url":"https://api.github.com/users/adamnovak","html_url":"https://github.com/adamnovak","followers_url":"https://api.github.com/users/adamnovak/followers","following_url":"https://api.github.com/users/adamnovak/following{/other_user}","gists_url":"https://api.github.com/users/adamnovak/gists{/gist_id}","starred_url":"https://api.github.com/users/adamnovak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adamnovak/subscriptions","organizations_url":"https://api.github.com/users/adamnovak/orgs","repos_url":"https://api.github.com/users/adamnovak/repos","events_url":"https://api.github.com/users/adamnovak/events{/privacy}","received_events_url":"https://api.github.com/users/adamnovak/received_events","type":"User","site_admin":false},"assignees":[{"login":"adamnovak","id":5062495,"node_id":"MDQ6VXNlcjUwNjI0OTU=","avatar_url":"https://avatars1.githubusercontent.com/u/5062495?v=4","gravatar_id":"","url":"https://api.github.com/users/adamnovak","html_url":"https://github.com/adamnovak","followers_url":"https://api.github.com/users/adamnovak/followers","following_url":"https://api.github.com/users/adamnovak/following{/other_user}","gists_url":"https://api.github.com/users/adamnovak/gists{/gist_id}","starred_url":"https://api.github.com/users/adamnovak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adamnovak/subscriptions","organizations_url":"https://api.github.com/users/adamnovak/orgs","repos_url":"https://api.github.com/users/adamnovak/repos","events_url":"https://api.github.com/users/adamnovak/events{/privacy}","received_events_url":"https://api.github.com/users/adamnovak/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/vgteam/vg/milestones/41","html_url":"https://github.com/vgteam/vg/milestone/41","labels_url":"https://api.github.com/repos/vgteam/vg/milestones/41/labels","id":3400433,"node_id":"MDk6TWlsZXN0b25lMzQwMDQzMw==","number":41,"title":"v1.9.0","description":"","creator":{"login":"adamnovak","id":5062495,"node_id":"MDQ6VXNlcjUwNjI0OTU=","avatar_url":"https://avatars1.githubusercontent.com/u/5062495?v=4","gravatar_id":"","url":"https://api.github.com/users/adamnovak","html_url":"https://github.com/adamnovak","followers_url":"https://api.github.com/users/adamnovak/followers","following_url":"https://api.github.com/users/adamnovak/following{/other_user}","gists_url":"https://api.github.com/users/adamnovak/gists{/gist_id}","starred_url":"https://api.github.com/users/adamnovak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adamnovak/subscriptions","organizations_url":"https://api.github.com/users/adamnovak/orgs","repos_url":"https://api.github.com/users/adamnovak/repos","events_url":"https://api.github.com/users/adamnovak/events{/privacy}","received_events_url":"https://api.github.com/users/adamnovak/received_events","type":"User","site_admin":false},"open_issues":5,"closed_issues":2,"state":"open","created_at":"2018-06-04T22:35:56Z","updated_at":"2018-06-13T19:48:11Z","due_on":"2018-07-16T07:00:00Z","closed_at":null},"comments":3,"created_at":"2018-06-13T19:48:11Z","updated_at":"2018-06-14T13:08:12Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"Since the safe SVs merge commit, if you use:\r\n\r\ns3://cgl-pipeline-inputs/vg_cgl/pop-map/graphs/wg38/v2/ALL.chrX_GRCh38.genotypes.20170504_minaf_0.1.vcf.gz\r\ns3://cgl-pipeline-inputs/vg_cgl/pop-map/graphs/wg38/v2/ALL.chrX_GRCh38.genotypes.20170504_minaf_0.1.vcf.gz.tbi\r\ns3://cgl-pipeline-inputs/vg_cgl/pop-map/input/GRCh38.fa.gz .\r\n\r\nAnd you do:\r\n```\r\ngunzip GRCh38.fa.gz\r\nvg construct -p --reference GRCh38.fa --vcf ALL.chrX_GRCh38.genotypes.20170504_minaf_0.1.vcf.gz --region X --region-is-chrom --node-max 32 --alt-paths --threads 1 >/dev/null\r\n```\r\n\r\nThere is a crash because we try to go beyond the end of the chunk when creating reference nodes, at about 55.2% done.\r\n\r\n```\r\n(gdb) bt\r\n#0  __cxxabiv1::__cxa_throw (obj=obj@entry=0x2489d40, tinfo=0x1970270 <typeinfo for std::out_of_range@@GLIBCXX_3.4>, \r\n    dest=0x6655b0 <std::out_of_range::~out_of_range()@plt>)\r\n    at /public/home/anovak/build/gcc-8.1.0/libstdc++-v3/libsupc++/eh_throw.cc:80\r\n#1  0x00007fe216510891 in std::__throw_out_of_range_fmt (__fmt=<optimized out>) from /public/home/anovak/.local/lib64/libstdc++.so.6\r\n#2  0x0000000000db28b9 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_check (\r\n    __s=0x143fd77 \"basic_string::substr\", __pos=<optimized out>, this=<optimized out>)\r\n    at /public/home/anovak/.local/include/c++/8.1.0/bits/char_traits.h:350\r\n#3  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::substr (__n=<optimized out>, \r\n    __pos=<optimized out>, this=<optimized out>) at /public/home/anovak/.local/include/c++/8.1.0/bits/basic_string.h:2807\r\n#4  vg::Constructor::construct_chunk(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<vcflib::Variant, std::allocator<vcflib::Variant> >, unsigned long) const::{lambda(unsigned long)#3}::operator()(unsigned long) const () at src/constructor.cpp:347\r\n#5  0x0000000000db563b in vg::Constructor::construct_chunk(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<vcflib::Variant, std::allocator<vcflib::Variant> >, unsigned long) const () at src/constructor.cpp:946\r\n#6  0x0000000000db852a in vg::Constructor::construct_graph(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, FastaReference&, vg::VcfBuffer&, std::vector<FastaReference*, std::allocator<FastaReference*> > const&, std::function<void (vg::Graph&)>) () at /public/home/anovak/.local/include/c++/8.1.0/new:169\r\n#7  0x0000000000db901d in vg::Constructor::construct_graph(std::vector<FastaReference*, std::allocator<FastaReference*> > const&, std::vector<vcflib::VariantCallFile*, std::allocator<vcflib::VariantCallFile*> > const&, std::vector<FastaReference*, std::allocator<FastaReference*> > const&, std::function<void (vg::Graph&)>) () at src/constructor.cpp:1644\r\n#8  0x0000000000b2fa83 in main_construct(int, char**) () at /public/home/anovak/.local/include/c++/8.1.0/new:169\r\n#9  0x0000000000b00ae8 in std::function<int (int, char**)>::operator()(int, char**) const (__args#1=<optimized out>, \r\n    __args#0=<optimized out>, this=<optimized out>) at /public/home/anovak/.local/include/c++/8.1.0/bits/std_function.h:260\r\n#10 vg::subcommand::Subcommand::operator() (this=<optimized out>, argc=argc@entry=15, argv=argv@entry=0x7fffa20cdde8)\r\n    at src/subcommand/subcommand.cpp:72\r\n#11 0x000000000073eeb7 in main () at src/main.cpp:61\r\n#12 0x00007fe215c93445 in __libc_start_main () from /usr/lib64/libc.so.6\r\n#13 0x00000000007f4eee in _start () at src/main.cpp:70\r\n```\r\n\r\nI can find the chunk position and the reference size for the chunk, and I see the reference cursor getting beyond the end of the chunk somehow.\r\n\r\n```\r\n(gdb) p reference_sequence.size()\r\n$2 = 521981\r\n(gdb) p reference_cursor\r\n$3 = 1107218\r\n(gdb) p chunk_offset\r\n$4 = 86107742\r\n```\r\n\r\nThe crash happens in b5b3b2f8279003f03ae2b91e7a193bfe41dd8fd0 but not in 62b2729b3e63954aea6bb6b76cbc7349c2aa60d4.\r\n"},"comment":{"url":"https://api.github.com/repos/vgteam/vg/issues/comments/397289622","html_url":"https://github.com/vgteam/vg/issues/1737#issuecomment-397289622","issue_url":"https://api.github.com/repos/vgteam/vg/issues/1737","id":397289622,"node_id":"MDEyOklzc3VlQ29tbWVudDM5NzI4OTYyMg==","user":{"login":"edawson","id":2595153,"node_id":"MDQ6VXNlcjI1OTUxNTM=","avatar_url":"https://avatars2.githubusercontent.com/u/2595153?v=4","gravatar_id":"","url":"https://api.github.com/users/edawson","html_url":"https://github.com/edawson","followers_url":"https://api.github.com/users/edawson/followers","following_url":"https://api.github.com/users/edawson/following{/other_user}","gists_url":"https://api.github.com/users/edawson/gists{/gist_id}","starred_url":"https://api.github.com/users/edawson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/edawson/subscriptions","organizations_url":"https://api.github.com/users/edawson/orgs","repos_url":"https://api.github.com/users/edawson/repos","events_url":"https://api.github.com/users/edawson/events{/privacy}","received_events_url":"https://api.github.com/users/edawson/received_events","type":"User","site_admin":false},"created_at":"2018-06-14T13:08:12Z","updated_at":"2018-06-14T13:08:12Z","author_association":"CONTRIBUTOR","body":"This SV has its END defined on GRCh37, which is probably what threw the cursor off.\r\n\r\nSorry to steamroll your fix, Glenn. If I remember correctly I was trying to recover the original output, maybe to pass the tests. I clearly didn't get it quite right.\r\n\r\nvg (before any safe-svs PRs) sees this and says it's not an SV (since it passes the allATGC check), and `variant_acceptable = true`, so it would get included in the graph (whether or not `-S` is passed).\r\n\r\nSV-aware (but broken) vg sees this is an SV (since it has an SVTYPE), so if we have `variant_acceptable = !is_sv()`, this variant doesn't get included without `-S`.\r\n\r\nSince the current batch of VCFs seems to prefer putting things in ALT alleles, my plan is to make those first priority and default to testing if something is an SV only after the allATGCN check fails. Does this sound alright to folks?\r\n\r\nI'll work on reducing the complexity/fragility of the SV handling code."}}
{"actor":{"display_login":"edawson"}}