{"action":"created","issue":{"url":"https://api.github.com/repos/spdk/spdk/issues/261","repository_url":"https://api.github.com/repos/spdk/spdk","labels_url":"https://api.github.com/repos/spdk/spdk/issues/261/labels{/name}","comments_url":"https://api.github.com/repos/spdk/spdk/issues/261/comments","events_url":"https://api.github.com/repos/spdk/spdk/issues/261/events","html_url":"https://github.com/spdk/spdk/issues/261","id":304268357,"number":261,"title":" Core dump occurred when run nvmf fio test","user":{"login":"WenqianZong","id":35021568,"avatar_url":"https://avatars1.githubusercontent.com/u/35021568?v=4","gravatar_id":"","url":"https://api.github.com/users/WenqianZong","html_url":"https://github.com/WenqianZong","followers_url":"https://api.github.com/users/WenqianZong/followers","following_url":"https://api.github.com/users/WenqianZong/following{/other_user}","gists_url":"https://api.github.com/users/WenqianZong/gists{/gist_id}","starred_url":"https://api.github.com/users/WenqianZong/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/WenqianZong/subscriptions","organizations_url":"https://api.github.com/users/WenqianZong/orgs","repos_url":"https://api.github.com/users/WenqianZong/repos","events_url":"https://api.github.com/users/WenqianZong/events{/privacy}","received_events_url":"https://api.github.com/users/WenqianZong/received_events","type":"User","site_admin":false},"labels":[{"id":235898962,"url":"https://api.github.com/repos/spdk/spdk/labels/bug","name":"bug","color":"fc2929","default":true}],"state":"open","locked":false,"assignee":{"login":"WenqianZong","id":35021568,"avatar_url":"https://avatars1.githubusercontent.com/u/35021568?v=4","gravatar_id":"","url":"https://api.github.com/users/WenqianZong","html_url":"https://github.com/WenqianZong","followers_url":"https://api.github.com/users/WenqianZong/followers","following_url":"https://api.github.com/users/WenqianZong/following{/other_user}","gists_url":"https://api.github.com/users/WenqianZong/gists{/gist_id}","starred_url":"https://api.github.com/users/WenqianZong/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/WenqianZong/subscriptions","organizations_url":"https://api.github.com/users/WenqianZong/orgs","repos_url":"https://api.github.com/users/WenqianZong/repos","events_url":"https://api.github.com/users/WenqianZong/events{/privacy}","received_events_url":"https://api.github.com/users/WenqianZong/received_events","type":"User","site_admin":false},"assignees":[{"login":"WenqianZong","id":35021568,"avatar_url":"https://avatars1.githubusercontent.com/u/35021568?v=4","gravatar_id":"","url":"https://api.github.com/users/WenqianZong","html_url":"https://github.com/WenqianZong","followers_url":"https://api.github.com/users/WenqianZong/followers","following_url":"https://api.github.com/users/WenqianZong/following{/other_user}","gists_url":"https://api.github.com/users/WenqianZong/gists{/gist_id}","starred_url":"https://api.github.com/users/WenqianZong/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/WenqianZong/subscriptions","organizations_url":"https://api.github.com/users/WenqianZong/orgs","repos_url":"https://api.github.com/users/WenqianZong/repos","events_url":"https://api.github.com/users/WenqianZong/events{/privacy}","received_events_url":"https://api.github.com/users/WenqianZong/received_events","type":"User","site_admin":false}],"milestone":null,"comments":3,"created_at":"2018-03-12T07:45:11Z","updated_at":"2018-05-23T06:30:06Z","closed_at":null,"author_association":"NONE","body":"**In https://review.gerrithub.io/#/c/403441/ patchset1\r\nnvmf target  core dumped \r\nfollowing is the information of core dump:** \r\n\r\n```\r\n00:51:44\t# gdb -batch -ex 'thread apply all bt full' ./app/nvmf_tgt/nvmf_tgt ./core.16876\r\n\r\nwarning: core file may not match specified executable file.\r\n[New LWP 19996]\r\n[New LWP 19995]\r\n[New LWP 19993]\r\n[New LWP 16876]\r\n[New LWP 19994]\r\n[New LWP 16928]\r\n[Thread debugging using libthread_db enabled]\r\nUsing host libthread_db library \"/lib64/libthread_db.so.1\".\r\nCore was generated by `./app/nvmf_tgt/nvmf_tgt -i 0 -c /home/sys_sgsw/build_pool/agent/repo/test/nvmf/'.\r\nProgram terminated with signal SIGABRT, Aborted.\r\n#0  0x00007f7ecc2269fb in raise () from /lib64/libc.so.6\r\n[Current thread is 1 (Thread 0x7f7ec698c700 (LWP 19996))]\r\n\r\nThread 6 (Thread 0x7f7ec8990700 (LWP 16928)):\r\n#0  0x00007f7ecc5d70f4 in recvmsg () from /lib64/libpthread.so.0\r\nNo symbol table info available.\r\n#1  0x0000000000546ffe in read_msg (m=0x7f7ec898d850, s=0x7f7ec898d7e0) at /home/sys_sgsw/build_pool/agent/repo/dpdk/lib/librte_eal/common/eal_common_proc.c:217\r\n        msglen = 0\r\n        iov = {iov_base = 0x7f7ec898d850, iov_len = 332}\r\n        msgh = {msg_name = 0x7f7ec898d7e0, msg_namelen = 110, msg_iov = 0x7f7ec898d7a0, msg_iovlen = 1, msg_control = 0x7f7ec898d730, msg_controllen = 48, msg_flags = 0}\r\n        control = '\\000' <repeats 47 times>\r\n        cmsg = 0x0\r\n        buflen = 332\r\n#2  0x00000000005472b5 in mp_handle (arg=0x0) at /home/sys_sgsw/build_pool/agent/repo/dpdk/lib/librte_eal/common/eal_common_proc.c:283\r\n        msg = {type = 0, msg = {name = '\\000' <repeats 63 times>, len_param = 0, num_fds = 0, param = '\\000' <repeats 255 times>, fds = {0, 0, 0, 0, 0, 0, 0, 0}}}\r\n        sa = {sun_family = 0, sun_path = '\\000' <repeats 107 times>}\r\n#3  0x00007f7ecc5cc36d in start_thread () from /lib64/libpthread.so.0\r\nNo symbol table info available.\r\n#4  0x00007f7ecc300b4f in clone () from /lib64/libc.so.6\r\nNo symbol table info available.\r\n\r\nThread 5 (Thread 0x7f7ec798e700 (LWP 19994)):\r\n#0  ibv_poll_cq (cq=0x7f7eb8000ac0, num_entries=32, wc=0x7f7ec798b0f0) at /usr/include/infiniband/verbs.h:1640\r\nNo locals.\r\n#1  0x00000000004d4bdf in spdk_nvmf_rdma_poller_poll (rtransport=0x2574210, rpoller=0x7f7eb8000a80) at rdma.c:1787\r\n        wc = {{wr_id = 140182229661280, status = IBV_WC_SUCCESS, opcode = IBV_WC_RDMA_WRITE, vendor_err = 0, byte_len = 4096, imm_data = 0, qp_num = 100, src_qp = 0, wc_flags = 0, pkey_index = 0, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182229661280, status = IBV_WC_SUCCESS, opcode = IBV_WC_SEND, vendor_err = 0, byte_len = 16, imm_data = 0, qp_num = 100, src_qp = 0, wc_flags = 0, pkey_index = 0, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182229660912, status = IBV_WC_SUCCESS, opcode = IBV_WC_RDMA_WRITE, vendor_err = 0, byte_len = 20480, imm_data = 0, qp_num = 100, src_qp = 0, wc_flags = 0, pkey_index = 0, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182229660912, status = IBV_WC_SUCCESS, opcode = IBV_WC_SEND, vendor_err = 0, byte_len = 16, imm_data = 0, qp_num = 100, src_qp = 0, wc_flags = 0, pkey_index = 0, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182229659072, status = IBV_WC_SUCCESS, opcode = IBV_WC_RDMA_WRITE, vendor_err = 0, byte_len = 36864, imm_data = 0, qp_num = 100, src_qp = 0, wc_flags = 0, pkey_index = 0, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182229659072, status = IBV_WC_SUCCESS, opcode = IBV_WC_SEND, vendor_err = 0, byte_len = 16, imm_data = 0, qp_num = 100, src_qp = 0, wc_flags = 0, pkey_index = 0, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182229641776, status = IBV_WC_SUCCESS, opcode = IBV_WC_SEND, vendor_err = 0, byte_len = 16, imm_data = 0, qp_num = 100, src_qp = 0, wc_flags = 0, pkey_index = 0, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182229642144, status = IBV_WC_SUCCESS, opcode = IBV_WC_SEND, vendor_err = 0, byte_len = 16, imm_data = 0, qp_num = 100, src_qp = 0, wc_flags = 0, pkey_index = 0, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182229626320, status = IBV_WC_SUCCESS, opcode = IBV_WC_SEND, vendor_err = 0, byte_len = 16, imm_data = 0, qp_num = 100, src_qp = 0, wc_flags = 0, pkey_index = 0, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182229626688, status = IBV_WC_SUCCESS, opcode = IBV_WC_SEND, vendor_err = 0, byte_len = 16, imm_data = 0, qp_num = 100, src_qp = 0, wc_flags = 0, pkey_index = 0, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182229627056, status = IBV_WC_SUCCESS, opcode = IBV_WC_SEND, vendor_err = 0, byte_len = 16, imm_data = 0, qp_num = 100, src_qp = 0, wc_flags = 0, pkey_index = 0, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182229627424, status = IBV_WC_SUCCESS, opcode = IBV_WC_SEND, vendor_err = 0, byte_len = 16, imm_data = 0, qp_num = 100, src_qp = 0, wc_flags = 0, pkey_index = 0, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182229624112, status = IBV_WC_SUCCESS, opcode = IBV_WC_SEND, vendor_err = 0, byte_len = 16, imm_data = 0, qp_num = 100, src_qp = 0, wc_flags = 0, pkey_index = 0, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182229671952, status = IBV_WC_SUCCESS, opcode = IBV_WC_RECV, vendor_err = 0, byte_len = 4160, imm_data = 2730468960, qp_num = 100, src_qp = 0, wc_flags = 1, pkey_index = 0, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182229624480, status = IBV_WC_SUCCESS, opcode = IBV_WC_SEND, vendor_err = 0, byte_len = 16, imm_data = 0, qp_num = 100, src_qp = 0, wc_flags = 0, pkey_index = 0, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182229670608, status = IBV_WC_SUCCESS, opcode = IBV_WC_RECV, vendor_err = 0, byte_len = 4160, imm_data = 2730468960, qp_num = 100, src_qp = 0, wc_flags = 1, pkey_index = 0, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182229641776, status = IBV_WC_SUCCESS, opcode = IBV_WC_SEND, vendor_err = 0, byte_len = 16, imm_data = 0, qp_num = 100, src_qp = 70923, wc_flags = 70922, pkey_index = 64640, slid = 65487, sl = 17 '\\021', dlid_path_bits = 0 '\\000'}, {wr_id = 140182229642144, status = 3428672169, opcode = 16809854, vendor_err = 3348673840, byte_len = 32638, imm_data = 5414903, qp_num = 0, src_qp = 0, wc_flags = 0, pkey_index = 100, slid = 1, sl = 99 'c', dlid_path_bits = 0 '\\000'}, {wr_id = 77303643264, status = 4291820544, opcode = 17, vendor_err = 3348673840, byte_len = 16809854, imm_data = 3348673888, qp_num = 32638, src_qp = 5570487, wc_flags = 0, pkey_index = 1, slid = 0, sl = 1 '\\001', dlid_path_bits = 0 '\\000'}, {wr_id = 140182491280848, status = 4291295104, opcode = 17, vendor_err = 0, byte_len = 65535, imm_data = 1, qp_num = 65634, src_qp = 65636, wc_flags = 65635, pkey_index = 1, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 281900178538596, status = 4289199232, opcode = 17, vendor_err = 3348673984, byte_len = 1, imm_data = 4289199488, qp_num = 17, src_qp = 65636, wc_flags = 131072, pkey_index = 1, slid = 0, sl = 198 '\\306', dlid_path_bits = 0 '\\000'}, {wr_id = 562945658454017, status = 3348673692, opcode = 32638, vendor_err = 3348673696, byte_len = 32638, imm_data = 3348673700, qp_num = 32638, src_qp = 65535, wc_flags = 0, pkey_index = 1, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 77303643136, status = IBV_WC_SUCCESS, opcode = IBV_WC_SEND, vendor_err = 0, byte_len = 0, imm_data = 0, qp_num = 1, src_qp = 3348674000, wc_flags = 32638, pkey_index = 64512, slid = 65447, sl = 17 '\\021', dlid_path_bits = 0 '\\000'}, {wr_id = 0, status = IBV_WC_SUCCESS, opcode = IBV_WC_RDMA_WRITE, vendor_err = 3348674000, byte_len = 32638, imm_data = 4289199104, qp_num = 17, src_qp = 3348673952, wc_flags = 32638, pkey_index = 32837, slid = 82, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182491280856, status = IBV_WC_SUCCESS, opcode = IBV_WC_RDMA_WRITE, vendor_err = 3348674000, byte_len = 32638, imm_data = 4291295104, qp_num = 17, src_qp = 3348673788, wc_flags = 32638, pkey_index = 31424, slid = 145, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182491280992, status = 5411108, opcode = IBV_WC_SEND, vendor_err = 4194400256, byte_len = 17, imm_data = 4291295104, qp_num = 17, src_qp = 1, wc_flags = 1, pkey_index = 1, slid = 0, sl = 1 '\\001', dlid_path_bits = 0 '\\000'}, {wr_id = 77208844288, status = 4291295104, opcode = IBV_WC_SEND, vendor_err = 4291295104, byte_len = 17, imm_data = 3348674000, qp_num = 32638, src_qp = 4291295104, wc_flags = 17, pkey_index = 0, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182491281296, status = 4293320464, opcode = 17, vendor_err = 4293320464, byte_len = 17, imm_data = 3348674448, qp_num = 32638, src_qp = 4236437696, wc_flags = 17, pkey_index = 54400, slid = 64642, sl = 17 '\\021', dlid_path_bits = 0 '\\000'}, {wr_id = 77250881728, status = 4236432512, opcode = 17, vendor_err = 3348674192, byte_len = 32638, imm_data = 3085627904, qp_num = 2736397881, src_qp = 3428648607, wc_flags = 32638, pkey_index = 8, slid = 0, sl = 0 '\\000', dlid_path_bits = 0 '\\000'}, {wr_id = 140182491281280, status = 4293320480, opcode = 17, vendor_err = 3087011136, byte_len = 32638, imm_data = 8, qp_num = 0, src_qp = 3348674432, wc_flags = 32638, pkey_index = 64512, slid = 65487, sl = 17 '\\021', dlid_path_bits = 0 '\\000'}, {wr_id = 140179142677772, status = IBV_WC_SUCCESS, opcode = 8, vendor_err = 0, byte_len = 1, imm_data = 1, qp_num = 1, src_qp = 0, wc_flags = 1, pkey_index = 8, slid = 0, sl = 12 '\\f', dlid_path_bits = 21 '\\025'}, {wr_id = 4294967297, status = IBV_WC_LOC_LEN_ERR, opcode = IBV_WC_RDMA_WRITE, vendor_err = 4293303808, byte_len = 17, imm_data = 3348674432, qp_num = 32638, src_qp = 4291820544, wc_flags = 17, pkey_index = 46976, slid = 51096, sl = 126 '~', dlid_path_bits = 127 '\\177'}}\r\n        rdma_req = 0x7f7eb800b660\r\n        rdma_recv = 0x7f7eb800f330\r\n        rqpair = 0x2570250\r\n        reaped = 0\r\n        i = 0\r\n        count = 0\r\n        error = false\r\n        __func__ = \"spdk_nvmf_rdma_poller_poll\"\r\n        __PRETTY_FUNCTION__ = \"spdk_nvmf_rdma_poller_poll\"\r\n#2  0x00000000004d5b3a in spdk_nvmf_rdma_poll_group_poll (group=0x7f7eb8000a50) at rdma.c:1878\r\n        rtransport = 0x2574210\r\n        rgroup = 0x7f7eb8000a50\r\n        rpoller = 0x7f7eb8000a80\r\n        count = 0\r\n        rc = 0\r\n#3  0x00000000004c6ee1 in spdk_nvmf_transport_poll_group_poll (group=0x7f7eb8000a50) at transport.c:163\r\nNo locals.\r\n#4  0x00000000004c2740 in spdk_nvmf_poll_group_poll (ctx=0x7f7eb80009b0) at nvmf.c:73\r\n        group = 0x7f7eb80009b0\r\n        rc = 0\r\n        tgroup = 0x7f7eb8000a50\r\n#5  0x00000000004ea190 in _spdk_reactor_run (arg=0x1a48700) at reactor.c:448\r\n        took_action = false\r\n        reactor = 0x1a48700\r\n        poller = 0x7f7eb8000a10\r\n        event_count = 0\r\n        idle_started = 0\r\n        now = 15190162968930\r\n        spin_cycles = 2199997\r\n        sleep_cycles = 2199997\r\n        sleep_us = 1000\r\n        timer_poll_count = 2\r\n        thread_name = \"reactor_1\", '\\000' <repeats 22 times>\r\n        __func__ = \"_spdk_reactor_run\"\r\n#6  0x00000000005397b3 in eal_thread_loop (arg=0x0) at /home/sys_sgsw/build_pool/agent/repo/dpdk/lib/librte_eal/linuxapp/eal/eal_thread.c:153\r\n        fct_arg = 0x1a48700\r\n        c = 0 '\\000'\r\n        n = 1\r\n        ret = 0\r\n        lcore_id = 1\r\n        thread_id = 140182491293440\r\n        m2s = 9\r\n        s2m = 12\r\n        cpuset = \"1\", '\\000' <repeats 254 times>\r\n        __func__ = \"eal_thread_loop\"\r\n#7  0x00007f7ecc5cc36d in start_thread () from /lib64/libpthread.so.0\r\nNo symbol table info available.\r\n#8  0x00007f7ecc300b4f in clone () from /lib64/libc.so.6\r\nNo symbol table info available.\r\n\r\nThread 4 (Thread 0x7f7ee1803c80 (LWP 16876)):\r\n#0  spdk_nvmf_poll_group_poll (ctx=0x1a48f10) at nvmf.c:72\r\n        group = 0x1a48f10\r\n        rc = 0\r\n        tgroup = 0x257c070\r\n#1  0x00000000004ea190 in _spdk_reactor_run (arg=0x1a48600) at reactor.c:448\r\n        took_action = false\r\n        reactor = 0x1a48600\r\n        poller = 0x1a47e70\r\n        event_count = 0\r\n        idle_started = 0\r\n        now = 15190162968088\r\n        spin_cycles = 2199997\r\n        sleep_cycles = 2199997\r\n        sleep_us = 1000\r\n        timer_poll_count = 4\r\n        thread_name = \"reactor_0\\000\\000\\000\\200\\000\\000\\000\\340J\\203G\\375\\177\\000\\000x-S\\000\\000\\000\\000\"\r\n        __func__ = \"_spdk_reactor_run\"\r\n#2  0x00000000004eb2e9 in spdk_reactors_start () at reactor.c:617\r\n        reactor = 0x1a48600\r\n        i = 4294967295\r\n        current_core = 0\r\n        rc = 0\r\n        __func__ = \"spdk_reactors_start\"\r\n        __PRETTY_FUNCTION__ = \"spdk_reactors_start\"\r\n#3  0x00000000004e7090 in spdk_app_start (opts=0x7ffd47834c40, start_fn=0x405e6e <nvmf_tgt_started>, arg1=0x0, arg2=0x0) at app.c:437\r\n        config = 0x1a41680\r\n        sp = 0x1a41650\r\n        shm_name = \"/nvmf_trace.0\", '\\000' <repeats 11 times>, \"\\377\\377\\377\\377\", '\\000' <repeats 20 times>, \"\\001\\000\\000\\000\", '\\377' <repeats 12 times>\r\n        rc = 0\r\n        tpoint_group_mask = 5609600\r\n        end = 0xffffffffffffffff <error: Cannot access memory at address 0xffffffffffffffff>\r\n        env_opts = {name = 0x55985d \"nvmf\", core_mask = 0x1a41a20 \"0xF\", shm_id = 0, mem_channel = -1, master_core = -1, mem_size = -1, no_pci = false, env_context = 0x0}\r\n        app_start_event = 0x11fc818000\r\n        __func__ = \"spdk_app_start\"\r\n        __PRETTY_FUNCTION__ = \"spdk_app_start\"\r\n#4  0x0000000000405fab in main (argc=5, argv=0x7ffd47834d88) at nvmf_main.c:80\r\n        rc = 1\r\n        opts = {name = 0x55985d \"nvmf\", config_file = 0x7ffd478358bc \"/home/sys_sgsw/build_pool/agent/repo/test/nvmf/fio/../nvmf.conf\", rpc_addr = 0x56f315 \"/var/tmp/spdk.sock\", reactor_mask = 0x1a41a20 \"0xF\", tpoint_group_mask = 0x0, shm_id = 0, shutdown_cb = 0x0, usr1_handler = 0x0, enable_coredump = true, mem_channel = -1, master_core = -1, mem_size = -1, no_pci = false, print_level = SPDK_LOG_NOTICE, max_delay_us = 1000}\r\n\r\nThread 3 (Thread 0x7f7ec818f700 (LWP 19993)):\r\n#0  0x00007f7ecc3010b3 in epoll_wait () from /lib64/libc.so.6\r\nNo symbol table info available.\r\n#1  0x000000000053dadf in eal_intr_handle_interrupts (pfd=21, totalfds=1) at /home/sys_sgsw/build_pool/agent/repo/dpdk/lib/librte_eal/linuxapp/eal/eal_interrupts.c:750\r\n        events = {{events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}}\r\n        nfds = 0\r\n#2  0x000000000053dcc0 in eal_intr_thread_main (arg=0x0) at /home/sys_sgsw/build_pool/agent/repo/dpdk/lib/librte_eal/linuxapp/eal/eal_interrupts.c:834\r\n        pipe_event = {events = 3, data = {ptr = 0x7, fd = 7, u32 = 7, u64 = 7}}\r\n        src = 0x0\r\n        numfds = 1\r\n        pfd = 21\r\n        ev = {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}\r\n        __func__ = \"eal_intr_thread_main\"\r\n#3  0x00007f7ecc5cc36d in start_thread () from /lib64/libpthread.so.0\r\nNo symbol table info available.\r\n#4  0x00007f7ecc300b4f in clone () from /lib64/libc.so.6\r\nNo symbol table info available.\r\n\r\nThread 2 (Thread 0x7f7ec718d700 (LWP 19995)):\r\n#0  0x000000000052aaa8 in __rte_ring_move_cons_head (entries=0x7f7ec718a698, new_head=0x7f7ec718a694, old_head=0x7f7ec718a690, behavior=RTE_RING_QUEUE_VARIABLE, n=8, is_sc=1, r=0x11ffc7f980) at /home/sys_sgsw/build_pool/agent/repo/dpdk/build/include/rte_ring_generic.h:153\r\n        prod_tail = 167940\r\n        max = 8\r\n        success = 1\r\n#1  __rte_ring_do_dequeue (available=0x0, is_sc=1, behavior=RTE_RING_QUEUE_VARIABLE, n=8, obj_table=0x7f7ec718a780, r=0x11ffc7f980) at /home/sys_sgsw/build_pool/agent/repo/dpdk/build/include/rte_ring.h:389\r\n        cons_head = 167940\r\n        cons_next = 32638\r\n        entries = 0\r\n#2  rte_ring_sc_dequeue_burst (available=0x0, n=8, obj_table=0x7f7ec718a780, r=0x11ffc7f980) at /home/sys_sgsw/build_pool/agent/repo/dpdk/build/include/rte_ring.h:912\r\nNo locals.\r\n#3  spdk_ring_dequeue (ring=0x11ffc7f980, objs=0x7f7ec718a780, count=8) at env.c:358\r\nNo locals.\r\n#4  0x00000000004e834b in _spdk_event_queue_run_batch (reactor=0x1a48800) at reactor.c:188\r\n        count = 0\r\n        i = 1\r\n        events = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}\r\n        __PRETTY_FUNCTION__ = \"_spdk_event_queue_run_batch\"\r\n#5  0x00000000004e9e94 in _spdk_reactor_run (arg=0x1a48800) at reactor.c:439\r\n        took_action = false\r\n        reactor = 0x1a48800\r\n        poller = 0x7f7eb0000a10\r\n        event_count = 0\r\n        idle_started = 0\r\n        now = 15190162999900\r\n        spin_cycles = 2199997\r\n        sleep_cycles = 2199997\r\n        sleep_us = 1000\r\n        timer_poll_count = 1\r\n        thread_name = \"reactor_2\", '\\000' <repeats 22 times>\r\n        __func__ = \"_spdk_reactor_run\"\r\n#6  0x00000000005397b3 in eal_thread_loop (arg=0x0) at /home/sys_sgsw/build_pool/agent/repo/dpdk/lib/librte_eal/linuxapp/eal/eal_thread.c:153\r\n        fct_arg = 0x1a48800\r\n        c = 0 '\\000'\r\n        n = 1\r\n        ret = 0\r\n        lcore_id = 2\r\n        thread_id = 140182482900736\r\n        m2s = 13\r\n        s2m = 16\r\n        cpuset = \"2\", '\\000' <repeats 254 times>\r\n        __func__ = \"eal_thread_loop\"\r\n#7  0x00007f7ecc5cc36d in start_thread () from /lib64/libpthread.so.0\r\nNo symbol table info available.\r\n#8  0x00007f7ecc300b4f in clone () from /lib64/libc.so.6\r\nNo symbol table info available.\r\n\r\nThread 1 (Thread 0x7f7ec698c700 (LWP 19996)):\r\n#0  0x00007f7ecc2269fb in raise () from /lib64/libc.so.6\r\nNo symbol table info available.\r\n#1  0x00007f7ecc228800 in abort () from /lib64/libc.so.6\r\nNo symbol table info available.\r\n#2  0x00007f7ecc26cbb1 in __libc_message () from /lib64/libc.so.6\r\nNo symbol table info available.\r\n#3  0x00007f7ecc277a59 in _int_free () from /lib64/libc.so.6\r\nNo symbol table info available.\r\n#4  0x00007f7ecc27d3be in free () from /lib64/libc.so.6\r\nNo symbol table info available.\r\n#5  0x00000000004f7263 in _spdk_io_device_attempt_free (dev=0x1a49090) at io_channel.c:361\r\n        thread = 0x0\r\n        ch = 0x0\r\n#6  0x00000000004f84ed in _spdk_put_io_channel (arg=0x7f7eb4000ba0) at io_channel.c:485\r\n        ch = 0x7f7eb4000ba0\r\n        __PRETTY_FUNCTION__ = \"_spdk_put_io_channel\"\r\n        __func__ = \"_spdk_put_io_channel\"\r\n#7  0x00000000004e858c in _spdk_reactor_msg_passed (arg1=0x4f7fe5 <_spdk_put_io_channel>, arg2=0x7f7eb4000ba0) at reactor.c:210\r\n        fn = 0x4f7fe5 <_spdk_put_io_channel>\r\n#8  0x00000000004e84d0 in _spdk_event_queue_run_batch (reactor=0x1a48900) at reactor.c:197\r\n        event = 0x11fc817880\r\n        count = 2\r\n        i = 0\r\n        events = {0x11fc817880, 0x11fc817c40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}\r\n        __PRETTY_FUNCTION__ = \"_spdk_event_queue_run_batch\"\r\n#9  0x00000000004e9e94 in _spdk_reactor_run (arg=0x1a48900) at reactor.c:439\r\n        took_action = false\r\n        reactor = 0x1a48900\r\n        poller = 0x7f7eb4000a10\r\n        event_count = 1\r\n        idle_started = 0\r\n        now = 15190160768317\r\n        spin_cycles = 2199997\r\n        sleep_cycles = 2199997\r\n        sleep_us = 1000\r\n        timer_poll_count = 4\r\n        thread_name = \"reactor_3\", '\\000' <repeats 22 times>\r\n        __func__ = \"_spdk_reactor_run\"\r\n#10 0x00000000005397b3 in eal_thread_loop (arg=0x0) at /home/sys_sgsw/build_pool/agent/repo/dpdk/lib/librte_eal/linuxapp/eal/eal_thread.c:153\r\n        fct_arg = 0x1a48900\r\n        c = 0 '\\000'\r\n        n = 1\r\n        ret = 0\r\n        lcore_id = 3\r\n        thread_id = 140182474508032\r\n        m2s = 17\r\n        s2m = 20\r\n        cpuset = \"3\", '\\000' <repeats 254 times>\r\n        __func__ = \"eal_thread_loop\"\r\n#11 0x00007f7ecc5cc36d in start_thread () from /lib64/libpthread.so.0\r\nNo symbol table info available.\r\n```\r\n\r\n**and https://review.gerrithub.io/#/c/390259/ \r\npatchset18 core dump is same as https://review.gerrithub.io/#/c/403441/\r\nfollowing is the information of core dump:**\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/spdk/spdk/issues/comments/391235218","html_url":"https://github.com/spdk/spdk/issues/261#issuecomment-391235218","issue_url":"https://api.github.com/repos/spdk/spdk/issues/261","id":391235218,"user":{"login":"wanqunintel","id":18566118,"avatar_url":"https://avatars3.githubusercontent.com/u/18566118?v=4","gravatar_id":"","url":"https://api.github.com/users/wanqunintel","html_url":"https://github.com/wanqunintel","followers_url":"https://api.github.com/users/wanqunintel/followers","following_url":"https://api.github.com/users/wanqunintel/following{/other_user}","gists_url":"https://api.github.com/users/wanqunintel/gists{/gist_id}","starred_url":"https://api.github.com/users/wanqunintel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wanqunintel/subscriptions","organizations_url":"https://api.github.com/users/wanqunintel/orgs","repos_url":"https://api.github.com/users/wanqunintel/repos","events_url":"https://api.github.com/users/wanqunintel/events{/privacy}","received_events_url":"https://api.github.com/users/wanqunintel/received_events","type":"User","site_admin":false},"created_at":"2018-05-23T06:30:06Z","updated_at":"2018-05-23T06:30:06Z","author_association":"NONE","body":"@WenqianZong can you try to reproduce it with Liang with latest code"}}
{"actor":{"display_login":"wanqunintel"}}