{"action":"created","issue":{"url":"https://api.github.com/repos/axboe/fio/issues/141","repository_url":"https://api.github.com/repos/axboe/fio","labels_url":"https://api.github.com/repos/axboe/fio/issues/141/labels{/name}","comments_url":"https://api.github.com/repos/axboe/fio/issues/141/comments","events_url":"https://api.github.com/repos/axboe/fio/issues/141/events","html_url":"https://github.com/axboe/fio/issues/141","id":126559743,"number":141,"title":"Opposite IO performance for numa setting","user":{"login":"gang2015","id":15097208,"avatar_url":"https://avatars3.githubusercontent.com/u/15097208?v=4","gravatar_id":"","url":"https://api.github.com/users/gang2015","html_url":"https://github.com/gang2015","followers_url":"https://api.github.com/users/gang2015/followers","following_url":"https://api.github.com/users/gang2015/following{/other_user}","gists_url":"https://api.github.com/users/gang2015/gists{/gist_id}","starred_url":"https://api.github.com/users/gang2015/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gang2015/subscriptions","organizations_url":"https://api.github.com/users/gang2015/orgs","repos_url":"https://api.github.com/users/gang2015/repos","events_url":"https://api.github.com/users/gang2015/events{/privacy}","received_events_url":"https://api.github.com/users/gang2015/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2016-01-14T02:01:23Z","updated_at":"2018-02-12T09:31:51Z","closed_at":null,"author_association":"NONE","body":"I setup a read test on a server with two numa nodes. fio runs random or sequential read with numa_cpu_node set 0 or 1 seperately.\n\nconfig example:\n[global]\noverwrite=1\ndirect=1\nioengine=libaio\nfilename=/dev/nvme1n1\nblocksize=4k\niodepth=1\ntime_based=1\nruntime=30m\n[read]\nnuma_cpu_nodes=1\nnuma_mem_policy=bind:1\nrw=read\n\nMy nvme drive is on PCIe with numa node =1. So I expect that there is be better performance for numa_cpu_nodes=1. During the test, I do see that fio process was running on the correct cpu core with numa bind. However, I get the opposite performance result: numa=1 always has much lower IOPS and higher latency. CPU load also shows higher on higher bandwidth. I can't explain the result. Is there anyone ran similar test before? \n\n<pre>\n                         Bandwidth (B)        IOPS          Latency 99.99%       Latency mean   CPU_load\nSeq read, bs=4k              189M              47.2K                  64              18            24%\niodepth=1,\nnuma_cpu_nodes=0\n\nSeq read, bs=4k              93M                23.8K                 85              37             15%\niodepth=1, \nnuma_cpu_nodes=1\n</pre>\n\n"},"comment":{"url":"https://api.github.com/repos/axboe/fio/issues/comments/364869529","html_url":"https://github.com/axboe/fio/issues/141#issuecomment-364869529","issue_url":"https://api.github.com/repos/axboe/fio/issues/141","id":364869529,"user":{"login":"gpkulkarni","id":7591634,"avatar_url":"https://avatars1.githubusercontent.com/u/7591634?v=4","gravatar_id":"","url":"https://api.github.com/users/gpkulkarni","html_url":"https://github.com/gpkulkarni","followers_url":"https://api.github.com/users/gpkulkarni/followers","following_url":"https://api.github.com/users/gpkulkarni/following{/other_user}","gists_url":"https://api.github.com/users/gpkulkarni/gists{/gist_id}","starred_url":"https://api.github.com/users/gpkulkarni/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gpkulkarni/subscriptions","organizations_url":"https://api.github.com/users/gpkulkarni/orgs","repos_url":"https://api.github.com/users/gpkulkarni/repos","events_url":"https://api.github.com/users/gpkulkarni/events{/privacy}","received_events_url":"https://api.github.com/users/gpkulkarni/received_events","type":"User","site_admin":false},"created_at":"2018-02-12T09:31:51Z","updated_at":"2018-02-12T09:31:51Z","author_association":"NONE","body":"any more findings/conclusions on this issue?"}}
{"actor":{"display_login":"gpkulkarni"}}