{"action":"created","issue":{"url":"https://api.github.com/repos/flux-framework/flux-sched/issues/109","labels_url":"https://api.github.com/repos/flux-framework/flux-sched/issues/109/labels{/name}","comments_url":"https://api.github.com/repos/flux-framework/flux-sched/issues/109/comments","events_url":"https://api.github.com/repos/flux-framework/flux-sched/issues/109/events","html_url":"https://github.com/flux-framework/flux-sched/pull/109","id":126787376,"number":109,"title":"rs2rank: add support to link resrc to broker rank and tests","user":{"login":"dongahn","id":862123,"avatar_url":"https://avatars.githubusercontent.com/u/862123?v=3","gravatar_id":"","url":"https://api.github.com/users/dongahn","html_url":"https://github.com/dongahn","followers_url":"https://api.github.com/users/dongahn/followers","following_url":"https://api.github.com/users/dongahn/following{/other_user}","gists_url":"https://api.github.com/users/dongahn/gists{/gist_id}","starred_url":"https://api.github.com/users/dongahn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dongahn/subscriptions","organizations_url":"https://api.github.com/users/dongahn/orgs","repos_url":"https://api.github.com/users/dongahn/repos","events_url":"https://api.github.com/users/dongahn/events{/privacy}","received_events_url":"https://api.github.com/users/dongahn/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":14,"created_at":"2016-01-15T01:30:10Z","updated_at":"2016-01-22T04:37:36Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/flux-framework/flux-sched/pulls/109","html_url":"https://github.com/flux-framework/flux-sched/pull/109","diff_url":"https://github.com/flux-framework/flux-sched/pull/109.diff","patch_url":"https://github.com/flux-framework/flux-sched/pull/109.patch"},"body":"Add support to correctly identify and target the controlling brokers to spawn a job, a fix needed to introduce necessary tests. In addition, this PR includes some refactoring work: i.e., introducing a bridge layer to hide normal vs. emulation scheduling mode as well as a bit better option/argument processing. \r\n\r\nOther changes are all testing related: \r\n* Add sched-once support to help eliminate race conditions in the new test cases I've added\r\n* Add and reorganize testing data including fake hwloc xml files\r\n* Add new test cases to test the basics of rs2rank and stress the system as well\r\n* Minor cleanup in some of the existing tests.\r\n\r\nNote that this PR includes fairly substantial changes, and I will appreciate if folks can help review this to minimize the errors that I probably have introduced. \r\n\r\nThe description of changes is captured in the commit messages. Commits in reverse chronological order:\r\n* rs2rank: add shardness test scripts and testing data\r\n* test: reorganize emulation testing data files\r\n* rs2rank: Identify and target the correct brokers to execute a job\r\n* schedsrv: introduce bridge layer to hide normal vs. emulation mode\r\n* schedsrv: fix a minor memory leak\r\n* schedsrv: add schedule once support for testing\r\n* schedsrv: refactor arguments parsing and handling\r\n\r\nI at least  made sure each commit builds and passes all tests on LLNL's cab machine. But this PR itself will fail in Travis because two pending `flux-core` PRs need to be merged into the core to be compatible with the latest changes. "},"comment":{"url":"https://api.github.com/repos/flux-framework/flux-sched/issues/comments/173804461","html_url":"https://github.com/flux-framework/flux-sched/pull/109#issuecomment-173804461","issue_url":"https://api.github.com/repos/flux-framework/flux-sched/issues/109","id":173804461,"user":{"login":"dongahn","id":862123,"avatar_url":"https://avatars.githubusercontent.com/u/862123?v=3","gravatar_id":"","url":"https://api.github.com/users/dongahn","html_url":"https://github.com/dongahn","followers_url":"https://api.github.com/users/dongahn/followers","following_url":"https://api.github.com/users/dongahn/following{/other_user}","gists_url":"https://api.github.com/users/dongahn/gists{/gist_id}","starred_url":"https://api.github.com/users/dongahn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dongahn/subscriptions","organizations_url":"https://api.github.com/users/dongahn/orgs","repos_url":"https://api.github.com/users/dongahn/repos","events_url":"https://api.github.com/users/dongahn/events{/privacy}","received_events_url":"https://api.github.com/users/dongahn/received_events","type":"User","site_admin":false},"created_at":"2016-01-22T04:37:36Z","updated_at":"2016-01-22T04:37:36Z","body":"@lipari: I've added some more debug print to debug why jsc and waitjob tests are failing on Travis. I believe it is failing because hwloc reader mode somehow truncates the hostname coming out of hwloc in building the resrc tree. \r\n\r\n```\r\n[1453436709.295181] sched.info[0]: hostname from hwloc testing-worker-linux-docker-006027f9-3209-linux-3\r\n\r\n<CUT> \r\n\r\n[1453436709.295248] sched.info[0]: resrc type: node, path: /cluster/testing-worker-linux-docker-6027, name: testing-worker-linux-docker-, digest: 82E447AD74E9D97BE1BC5E4D194D642F03901B7D, id: 6027, state: invalid, uuid: f1afe77e-8434-4230-a6d5-54616a287a64, size: 1, avail: 1\r\n```\r\n\r\nSince the tests are run under a docker container in Travis, the hostname turns out to be pretty long and it seems there is a logic somewhere that split the hostname into \"name\" and \"id\" for an resrc object, which doesn't quite work well for such a convoluted hostname.\r\n\r\nThe reason this problem only shows up now with my PR is that this PR discards the rdl-specifed resrc tree if it doesn't match with the runtime hwloc data and rebuilds the resrc tree with the hwloc data. So this is the first time where we tested jsc and waitjob tests with hwloc on Travis...\r\n\r\nOther new rs2rank tests work because they use fake hwloc xml files taken from cab and thus have well-formed hostname.\r\n\r\n\r\n"}}