{"action":"created","issue":{"url":"https://api.github.com/repos/softdevteam/krun/issues/192","repository_url":"https://api.github.com/repos/softdevteam/krun","labels_url":"https://api.github.com/repos/softdevteam/krun/issues/192/labels{/name}","comments_url":"https://api.github.com/repos/softdevteam/krun/issues/192/comments","events_url":"https://api.github.com/repos/softdevteam/krun/issues/192/events","html_url":"https://github.com/softdevteam/krun/pull/192","id":148080208,"number":192,"title":"Use CPU pinning on a set of CPUs","user":{"login":"vext01","id":604955,"avatar_url":"https://avatars.githubusercontent.com/u/604955?v=3","gravatar_id":"","url":"https://api.github.com/users/vext01","html_url":"https://github.com/vext01","followers_url":"https://api.github.com/users/vext01/followers","following_url":"https://api.github.com/users/vext01/following{/other_user}","gists_url":"https://api.github.com/users/vext01/gists{/gist_id}","starred_url":"https://api.github.com/users/vext01/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vext01/subscriptions","organizations_url":"https://api.github.com/users/vext01/orgs","repos_url":"https://api.github.com/users/vext01/repos","events_url":"https://api.github.com/users/vext01/events{/privacy}","received_events_url":"https://api.github.com/users/vext01/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/softdevteam/krun/labels/bug","name":"bug","color":"fc2929"},{"url":"https://api.github.com/repos/softdevteam/krun/labels/high%20priority%20(must%20have%20before%20publication)","name":"high priority (must have before publication)","color":"e11d21"}],"state":"open","locked":false,"assignee":{"login":"snim2","id":97674,"avatar_url":"https://avatars.githubusercontent.com/u/97674?v=3","gravatar_id":"","url":"https://api.github.com/users/snim2","html_url":"https://github.com/snim2","followers_url":"https://api.github.com/users/snim2/followers","following_url":"https://api.github.com/users/snim2/following{/other_user}","gists_url":"https://api.github.com/users/snim2/gists{/gist_id}","starred_url":"https://api.github.com/users/snim2/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/snim2/subscriptions","organizations_url":"https://api.github.com/users/snim2/orgs","repos_url":"https://api.github.com/users/snim2/repos","events_url":"https://api.github.com/users/snim2/events{/privacy}","received_events_url":"https://api.github.com/users/snim2/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/softdevteam/krun/milestones/2","html_url":"https://github.com/softdevteam/krun/milestones/camera-ready","labels_url":"https://api.github.com/repos/softdevteam/krun/milestones/2/labels","id":1396000,"number":2,"title":"camera-ready","description":"","creator":{"login":"vext01","id":604955,"avatar_url":"https://avatars.githubusercontent.com/u/604955?v=3","gravatar_id":"","url":"https://api.github.com/users/vext01","html_url":"https://github.com/vext01","followers_url":"https://api.github.com/users/vext01/followers","following_url":"https://api.github.com/users/vext01/following{/other_user}","gists_url":"https://api.github.com/users/vext01/gists{/gist_id}","starred_url":"https://api.github.com/users/vext01/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vext01/subscriptions","organizations_url":"https://api.github.com/users/vext01/orgs","repos_url":"https://api.github.com/users/vext01/repos","events_url":"https://api.github.com/users/vext01/events{/privacy}","received_events_url":"https://api.github.com/users/vext01/received_events","type":"User","site_admin":false},"open_issues":8,"closed_issues":11,"state":"open","created_at":"2015-11-05T16:48:44Z","updated_at":"2016-04-14T13:38:05Z","due_on":null,"closed_at":null},"comments":6,"created_at":"2016-04-13T14:32:13Z","updated_at":"2016-04-14T15:19:03Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/softdevteam/krun/pulls/192","html_url":"https://github.com/softdevteam/krun/pull/192","diff_url":"https://github.com/softdevteam/krun/pull/192.diff","patch_url":"https://github.com/softdevteam/krun/pull/192.patch"},"body":"Talks with Intel suggest that we should pin our benchmarks to a *set* of isolated cores. For us it makes a great deal of sense to isolate and pin to our adaptive ticks CPUs (all but the boot CPU due to `NO_HZ_FULL_ALL`).\r\n\r\nThis set of commits:\r\n\r\n * Ensures the adaptive ticks CPUs are isolated and guides the user if not.\r\n * Uses taskset to pin benchmarks with the correct affinity mask.\r\n * Tests all of this.\r\n * Sanity checks the affinity of benchmarks with a C program.\r\n * Updates related documentation.\r\n\r\nThe code was loosely based upon the previously reverted pinning code, where we pinned to one CPU only.\r\n\r\nAfter all of this I ran a jruby benchmark and used top to check how the threads are being scheduled. Here is how it looks:\r\n\r\n![affin2](https://cloud.githubusercontent.com/assets/604955/14497170/d20d5496-018d-11e6-88c4-ce90cd5a89f1.png)\r\n\r\n\r\nThe benchmark threads have the correct affinity, and are on an adaptive ticks CPU, but all threads run on the same CPU. I think we were expecting the threads to be spread out over the CPUs in the affinity mask (all CPUs apart from CPU 0). The left hand pane in the screen shot is showing that the affinity mask is correctly set.\r\n\r\nHaving seen this, I wondered if taskset has caused all threads to all go on one core. I commented out the use of taskset and the accompanying sanity check and re-ran. This time all threads are scheduled on the boot processor.\r\n\r\nWhat this means is that JRuby does not and has never utilised >1 CPU despite having loads of threads. Further, I can't say why removing pinning with a single CPU in the affinity mask (as we did a little while back) gave a speed-up.\r\n\r\nRaising the PR for code review and discussion. Please don't merge yet.\r\n\r\nNext port of call: write a small C program which spawns a load of threads. I want to know if threads simply default to being scheduled on the same CPU as their parent, or if the JVM is forcing all the threads on one core somehow.\r\n\r\nThanks\r\n\r\ncc @ltratt for discussion"},"comment":{"url":"https://api.github.com/repos/softdevteam/krun/issues/comments/209998343","html_url":"https://github.com/softdevteam/krun/pull/192#issuecomment-209998343","issue_url":"https://api.github.com/repos/softdevteam/krun/issues/192","id":209998343,"user":{"login":"vext01","id":604955,"avatar_url":"https://avatars.githubusercontent.com/u/604955?v=3","gravatar_id":"","url":"https://api.github.com/users/vext01","html_url":"https://github.com/vext01","followers_url":"https://api.github.com/users/vext01/followers","following_url":"https://api.github.com/users/vext01/following{/other_user}","gists_url":"https://api.github.com/users/vext01/gists{/gist_id}","starred_url":"https://api.github.com/users/vext01/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vext01/subscriptions","organizations_url":"https://api.github.com/users/vext01/orgs","repos_url":"https://api.github.com/users/vext01/repos","events_url":"https://api.github.com/users/vext01/events{/privacy}","received_events_url":"https://api.github.com/users/vext01/received_events","type":"User","site_admin":false},"created_at":"2016-04-14T15:19:03Z","updated_at":"2016-04-14T15:19:03Z","body":"OK to start squashing?"}}