{"action":"created","issue":{"url":"https://api.github.com/repos/scylladb/scylla/issues/3205","repository_url":"https://api.github.com/repos/scylladb/scylla","labels_url":"https://api.github.com/repos/scylladb/scylla/issues/3205/labels{/name}","comments_url":"https://api.github.com/repos/scylladb/scylla/issues/3205/comments","events_url":"https://api.github.com/repos/scylladb/scylla/issues/3205/events","html_url":"https://github.com/scylladb/scylla/issues/3205","id":296368959,"number":3205,"title":"Abort during wide-rows read workload","user":{"login":"denesb","id":1389273,"avatar_url":"https://avatars2.githubusercontent.com/u/1389273?v=4","gravatar_id":"","url":"https://api.github.com/users/denesb","html_url":"https://github.com/denesb","followers_url":"https://api.github.com/users/denesb/followers","following_url":"https://api.github.com/users/denesb/following{/other_user}","gists_url":"https://api.github.com/users/denesb/gists{/gist_id}","starred_url":"https://api.github.com/users/denesb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/denesb/subscriptions","organizations_url":"https://api.github.com/users/denesb/orgs","repos_url":"https://api.github.com/users/denesb/repos","events_url":"https://api.github.com/users/denesb/events{/privacy}","received_events_url":"https://api.github.com/users/denesb/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-02-12T13:07:46Z","updated_at":"2018-02-12T13:18:46Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"*Installation details*\r\nScylla version (or git commit hash): 666.development-0.20180212.29805da9f\r\nCluster size: 3\r\nOS (RHEL/CentOS/Ubuntu/AWS AMI): GCE/Centos7\r\nReport UUID:\r\n\r\nSame wide-rows workload as for #3189.\r\n\r\nBacktrace:\r\n```\r\n[New LWP 3619]\r\n[New LWP 3624]\r\n[New LWP 3621]\r\n[New LWP 3626]\r\n[New LWP 3623]\r\n[New LWP 3627]\r\n[New LWP 3628]\r\n[New LWP 3618]\r\n[New LWP 3625]\r\n[New LWP 3620]\r\n[New LWP 3622]\r\n[New LWP 3617]\r\nReading symbols from /usr/bin/scylla...Reading symbols from /usr/lib/debug/usr/bin/scylla.debug...done.\r\ndone.\r\n[Thread debugging using libthread_db enabled]\r\nUsing host libthread_db library \"/lib64/libthread_db.so.1\".\r\nCore was generated by `/usr/bin/scylla --log-to-syslog 1 --log-to-stdout 0 --default-log-level info --'.\r\nProgram terminated with signal SIGABRT, Aborted.\r\n#0  0x00007fe73983e1f7 in raise () from /lib64/libc.so.6\r\n[Current thread is 1 (Thread 0x7fe734fff700 (LWP 3619))]\r\n#0  0x00007fe73983e1f7 in raise () from /lib64/libc.so.6\r\n#1  0x00007fe73983fa28 in abort () from /lib64/libc.so.6\r\n#2  0x0000000000e9afdc in seastar::future<>::then_wrapped<dirty_memory_manager::flush_one(memtable_list&, flush_permit&&)::<lambda(auto:175)> > (func=<optimized out>, this=0x7fe734fd4520) at seastar/core/future.hh:980\r\n#3  dirty_memory_manager::flush_one(memtable_list&, flush_permit&&) (this=this@entry=0x6020001b7028, mtlist=..., permit=permit@entry=<unknown type in /usr/lib/debug/usr/bin/scylla.debug, CU 0x4d4d339, DIE 0x5eeca35>) at database.cc:3352\r\n#4  0x0000000000f7f80c in dirty_memory_manager::<lambda()>::<lambda()>::<lambda(auto:176)>::operator()<flush_permit> (permit=..., __closure=<optimized out>) at database.cc:3385\r\n#5  seastar::apply_helper<dirty_memory_manager::flush_when_needed()::<lambda()>::<lambda()>::<lambda(auto:176)>, std::tuple<flush_permit>&&, std::integer_sequence<long unsigned int, 0> >::apply (\r\n    args=<unknown type in /usr/lib/debug/usr/bin/scylla.debug, CU 0x4d4d339, DIE 0x6241793>, func=<optimized out>) at seastar/core/apply.hh:36\r\n#6  seastar::apply<dirty_memory_manager::flush_when_needed()::<lambda()>::<lambda()>::<lambda(auto:176)>, flush_permit> (args=<optimized out>, func=<optimized out>) at seastar/core/apply.hh:44\r\n#7  seastar::futurize<seastar::future<> >::apply<dirty_memory_manager::flush_when_needed()::<lambda()>::<lambda()>::<lambda(auto:176)>, flush_permit> (args=<unknown type in /usr/lib/debug/usr/bin/scylla.debug, CU 0x4d4d339, DIE 0x624174b>, \r\n    func=<optimized out>) at seastar/core/future.hh:1352\r\n#8  seastar::future<flush_permit>::then<dirty_memory_manager::flush_when_needed()::<lambda()>::<lambda()>::<lambda(auto:176)> > (func=<optimized out>, this=0x7fe734fd4740) at seastar/core/future.hh:922\r\n#9  dirty_memory_manager::<lambda()>::<lambda()>::operator() (__closure=0x7fe734fd47d0, __closure@entry=0x7fe734fd46a0) at database.cc:3388\r\n#10 seastar::apply_helper<dirty_memory_manager::flush_when_needed()::<lambda()>::<lambda()>, std::tuple<>&&, std::integer_sequence<long unsigned int> >::apply (args=<optimized out>, func=<optimized out>) at seastar/core/apply.hh:36\r\n#11 seastar::apply<dirty_memory_manager::flush_when_needed()::<lambda()>::<lambda()> > (args=<optimized out>, func=<optimized out>) at seastar/core/apply.hh:44\r\n#12 seastar::futurize<seastar::future<> >::apply<dirty_memory_manager::flush_when_needed()::<lambda()>::<lambda()> >(<unknown type in /usr/lib/debug/usr/bin/scylla.debug, CU 0x4d4d339, DIE 0x623e6d8>, <unknown type in /usr/lib/debug/usr/bin/scylla.debug, CU 0x4d4d339, DIE 0x623e6e1>) (func=func@entry=<unknown type in /usr/lib/debug/usr/bin/scylla.debug, CU 0x4d4d339, DIE 0x623e6d8>, args=<optimized out>) at seastar/core/future.hh:1352\r\n#13 0x0000000000f81944 in seastar::future<>::then<dirty_memory_manager::flush_when_needed()::<lambda()>::<lambda()> > (func=<unknown type in /usr/lib/debug/usr/bin/scylla.debug, CU 0x4d4d339, DIE 0x5ee7b99>, this=0x7fe734fd4800)\r\n    at seastar/core/future.hh:922\r\n#14 dirty_memory_manager::<lambda()>::operator()(void) const (__closure=__closure@entry=0x602000057590) at database.cc:3388\r\n#15 0x0000000000f82e1c in seastar::internal::do_until_state<dirty_memory_manager::flush_when_needed()::<lambda()>, dirty_memory_manager::flush_when_needed()::<lambda()> >::run_and_dispose(void) (this=0x602000057550)\r\n    at /usr/src/debug/scylla-666.development/seastar/core/future-util.hh:464\r\n#16 0x00000000005889c7 in seastar::reactor::run_tasks (this=this@entry=0x602000003000, tq=...) at core/reactor.cc:2495\r\n#17 0x0000000000588b8a in seastar::reactor::run_some_tasks (this=this@entry=0x602000003000, t_run_completed=...) at core/reactor.cc:2893\r\n#18 0x0000000000660d0d in seastar::reactor::run_some_tasks (t_run_completed=..., this=0x602000003000) at /opt/scylladb/include/c++/7/chrono:377\r\n#19 seastar::reactor::run (this=0x602000003000) at core/reactor.cc:3035\r\n#20 0x0000000000667674 in seastar::smp::<lambda()>::operator()(void) const (__closure=<optimized out>) at core/reactor.cc:3983\r\n#21 0x0000000000743a8e in std::function<void ()>::operator()() const (this=<optimized out>) at /opt/scylladb/include/c++/7/bits/std_function.h:706\r\n#22 seastar::posix_thread::start_routine (arg=<optimized out>) at core/posix.cc:52\r\n#23 0x00007fe739bd3e25 in start_thread () from /lib64/libpthread.so.0\r\n#24 0x00007fe73990134d in clone () from /lib64/libc.so.6\r\n```"},"comment":{"url":"https://api.github.com/repos/scylladb/scylla/issues/comments/364919925","html_url":"https://github.com/scylladb/scylla/issues/3205#issuecomment-364919925","issue_url":"https://api.github.com/repos/scylladb/scylla/issues/3205","id":364919925,"user":{"login":"nyh","id":584227,"avatar_url":"https://avatars3.githubusercontent.com/u/584227?v=4","gravatar_id":"","url":"https://api.github.com/users/nyh","html_url":"https://github.com/nyh","followers_url":"https://api.github.com/users/nyh/followers","following_url":"https://api.github.com/users/nyh/following{/other_user}","gists_url":"https://api.github.com/users/nyh/gists{/gist_id}","starred_url":"https://api.github.com/users/nyh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nyh/subscriptions","organizations_url":"https://api.github.com/users/nyh/orgs","repos_url":"https://api.github.com/users/nyh/repos","events_url":"https://api.github.com/users/nyh/events{/privacy}","received_events_url":"https://api.github.com/users/nyh/received_events","type":"User","site_admin":false},"created_at":"2018-02-12T13:18:46Z","updated_at":"2018-02-12T13:18:46Z","author_association":"CONTRIBUTOR","body":"Yes, indeed it is out of memory which probably indicates a bug that consumed all memory. But still it would have been nice to fail the memory-hogging operation - not crash the entire system (this is https://github.com/scylladb/seastar/issues/84)."}}
{"actor":{"display_login":"nyh"}}