{"action":"opened","issue":{"url":"https://api.github.com/repos/dmwm/WMCore/issues/8646","repository_url":"https://api.github.com/repos/dmwm/WMCore","labels_url":"https://api.github.com/repos/dmwm/WMCore/issues/8646/labels{/name}","comments_url":"https://api.github.com/repos/dmwm/WMCore/issues/8646/comments","events_url":"https://api.github.com/repos/dmwm/WMCore/issues/8646/events","html_url":"https://github.com/dmwm/WMCore/issues/8646","id":330594108,"node_id":"MDU6SXNzdWUzMzA1OTQxMDg=","number":8646,"title":"Create new API for job requirements update on the fly","user":{"login":"amaltaro","id":1798520,"node_id":"MDQ6VXNlcjE3OTg1MjA=","avatar_url":"https://avatars0.githubusercontent.com/u/1798520?v=4","gravatar_id":"","url":"https://api.github.com/users/amaltaro","html_url":"https://github.com/amaltaro","followers_url":"https://api.github.com/users/amaltaro/followers","following_url":"https://api.github.com/users/amaltaro/following{/other_user}","gists_url":"https://api.github.com/users/amaltaro/gists{/gist_id}","starred_url":"https://api.github.com/users/amaltaro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amaltaro/subscriptions","organizations_url":"https://api.github.com/users/amaltaro/orgs","repos_url":"https://api.github.com/users/amaltaro/repos","events_url":"https://api.github.com/users/amaltaro/events{/privacy}","received_events_url":"https://api.github.com/users/amaltaro/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-06-08T09:53:35Z","updated_at":"2018-06-08T09:53:35Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"Seangchan and I discussed this new feature a few weeks ago and it consists of providing a ReqMgr2 API that would allow a few parameters - job requirements wise - to be updated while the workflow is running. \r\nThe expected behavior is:\r\n1) ReqMgr2 itself won't do anything automatically, it needs an external system to post data to this new API.\r\n2) it won't touch jobs already created in the agents (and WQE in Running state or beyond). We can further discuss this and address it in different ways (not here though).\r\n3) the initial allowed parameters are: Memory (thus MaxRSS), Multicore and TimePerEvent (job splitting)\r\n4) this API only accepts updates for workflows in running-* state\r\n\r\nImplementation details are:\r\n*) workflows will have a timestamp for the last modification (after it gets acquired, for any non-status modification). How we handle RequestPriority changes?\r\n*) a new wmbs_workflow_ts table willl be created. With 2 columns, top level workflow id (from wmbs_workflow?) and timestamp in which the workflow sandbox was created (last modified)\r\n*) a couch view has to be created for workflow and timestamp fields\r\n*) JobUpdater will look up at all workflows in this table - we need to somehow parse workflows that are in running state only - and compare the timestamp against the request in ReqMgr2 (using couch view). In case an update is needed: \r\n  *) the spec inside the sandbox has to be updated (probably better if we simply retrieve the new one from ReqMgr2\r\n  *) then update wmbs_workflow_ts row\r\n  *) it seems we don't need to make any updates to the LQE and GQE\r\n*) all these updates must be cleared recorded in the logs (and/or maybe in logDB)\r\n\r\nPossible collateral issues are:\r\n*) top level job estimates (global workqueue) will be wrong, but I guess nobody actually cares\r\n*) we need to make sure these updates are easily visible, to avoid problems in wf debugging.\r\n*) anything else?\r\n\r\n@ticoann let me know if I missed something\r\n\r\n"}}
{"actor":{"display_login":"amaltaro"}}