{"push_id":945491417,"size":6,"distinct_size":6,"ref":"refs/heads/rs2rank_ts","head":"8ac2017413c6613966ac0d0019f6d3005ef24462","before":"1dba9c2c24bfaa6840f59b4b3aa38399f7885b09","commits":[{"sha":"b80fc87033c651ba7fc1a99f5079e15ad130dd1b","author":{"email":"723cba40ab9a9bbf5106ca015d08171aa32f176a@llnl.gov","name":"Dong H. Ahn"},"message":"rs2rank: Identify and target the correct brokers to execute a job\n\nFix issue #69 where `schedsrv` incorrectly identified and targeted\nthe first k consecutive broker ranks (starting from rank 0) to launch\na job allocated to any k number of nodes: that is, regardless of\nwhat nodes it is allocated to and what broker ranks manage and\ncontrol these nodes.\nFor example, even if two jobs are allocated back to back to\n`cab[1-2]` and `cab[3-4]`, the old scheme will end up launching\nboth jobs into `cab[1-2]`, assuming rank 0 and 1\nare running on these two nodes.\n\nTo fix this while reducing code complexity, this commit also abstracts\nout the multiple resource-reader-option and resource-to-rank\nmapping codes into two separate classes: i.e., `rsreader` and `rs2rank`.\n\nSpecifically, as part of `load_resources ()`, `schedsrv` now\nuses `rsreader`'s resource load API and populate the resrc hierarchy\nand also use `rs2rank` APIs to link each node-type `resrc_t` object\nin this hierarchy to its managing broker rank(s).\n\nThe main data structure that assists schedsrv with this linking\nis a rs2rank table (of `machs_t` type).\nIt is a hash table keyed by a hostname, and each table entry\nmaintains the signature of each per-rank resource partition\n(e.g., `core[0-7]` subset of `core[0-15]` node) within this host\nas well as one or more equivalent broker ranks that manage\nand control the resource partition.\n\nNext, we identify each per-rank resource partition by taking the SHA\ndigest of per-rank hwloc xml data that resides in\n`resource.hwloc.xml.<rank>` in kvs. If two brokers on the same\nhost happen to have the identical xml data under this key, they are\ntreated as equivalent ranks.\n\n`resrc_t` class is also modified such that each node-type\nresrc object gets filled in with the per-rank signature during\nthe linking phase. Once linked, its hostname and signature\nare used to uniquely identify the ranks set that can/should spawn\nthe job processes on the nodes allocated to it.\n\nDuring the linking process, we also perform a consistency\ncheck by comparing the constructed resrc hierarchy with the\nactual hwloc data by using `rs2rank` API as well.\nIf inconsistency is detected, we print out an error message and\nsimply discard the existing hierarchy and rebuild a new one\nbased on the hwloc xml data, instead.\n\nFinally for emulation mode (`RSREADER_RESRC_EMUL`), because\nthe rdl and hwloc xml data would be almost always inconsistent,\nwe simply force a link to associate all of the node-type\n`resrc_t` objects with rank 0. This can easily be extended\nto support fake hwloc mode for scalability testing beyond\nemulation.\n\nClose #69","distinct":true,"url":"https://api.github.com/repos/dongahn/flux-sched/commits/b80fc87033c651ba7fc1a99f5079e15ad130dd1b"},{"sha":"ed5115586fcc81108b2aa331ce4ec5577367927c","author":{"email":"723cba40ab9a9bbf5106ca015d08171aa32f176a@llnl.gov","name":"Dong H. Ahn"},"message":"test: reorganize emulation testing data files\n\nMove the emulation shareness testing files to extend\n`sched` test further.\n\n`t/data` is now the root directory that will contain\ntesting specific data including fake hwloc data files.\n\nAdd some minor cleanup to emulation tests.","distinct":true,"url":"https://api.github.com/repos/dongahn/flux-sched/commits/ed5115586fcc81108b2aa331ce4ec5577367927c"},{"sha":"2606e5ec3a4826c9848725974e4b6adb3467505a","author":{"email":"723cba40ab9a9bbf5106ca015d08171aa32f176a@llnl.gov","name":"Dong H. Ahn"},"message":"rs2rank: add rs2rank testing data\n\n`t/data/hwloc-data` contains all of the hwloc xml data and\nsome rdl files (that allow testing of matching vs. unmatching\nspecification).\n\nAn upcoming rs2rank sharness tests will use these data.","distinct":true,"url":"https://api.github.com/repos/dongahn/flux-sched/commits/2606e5ec3a4826c9848725974e4b6adb3467505a"},{"sha":"7b43c954a252116c5d9e1c0be0e187210d6f3bf2","author":{"email":"723cba40ab9a9bbf5106ca015d08171aa32f176a@llnl.gov","name":"Dong H. Ahn"},"message":"rs2rank: add shareness test scripts\n\nAdd shareness tests that utilize a new hwloc front-end command\n(`flux hwloc`) within `flux-core` to load in synthentic hwloc xml files\nand submit and run a bunch of unit jobs and verify the\nresults (e.g., the correct rank spawned the job.)\n\nThe description of each test should be self-exploratory\nby means of the name of the script and printing message of\neach subtest.","distinct":true,"url":"https://api.github.com/repos/dongahn/flux-sched/commits/7b43c954a252116c5d9e1c0be0e187210d6f3bf2"},{"sha":"2679ac318a19a061c45bc6671d1e0f24c7c461a0","author":{"email":"723cba40ab9a9bbf5106ca015d08171aa32f176a@llnl.gov","name":"Dong H. Ahn"},"message":"travis: now print logs if a test fails","distinct":true,"url":"https://api.github.com/repos/dongahn/flux-sched/commits/2679ac318a19a061c45bc6671d1e0f24c7c461a0"},{"sha":"8ac2017413c6613966ac0d0019f6d3005ef24462","author":{"email":"723cba40ab9a9bbf5106ca015d08171aa32f176a@llnl.gov","name":"Dong H. Ahn"},"message":"schedsrv: add verbosity flag to sched module\n\nAdd verbosity level switch, verbosity=<num>, to the load line of the sched module.\nIt is currently used to avoid printing of the resrc tree state all the time\nwith DEBUG_LOG. It can later be used to print debug or info\nmessages with different level of details for sched module.","distinct":true,"url":"https://api.github.com/repos/dongahn/flux-sched/commits/8ac2017413c6613966ac0d0019f6d3005ef24462"}]}