{"action":"closed","issue":{"url":"https://api.github.com/repos/flux-framework/flux-core/issues/614","repository_url":"https://api.github.com/repos/flux-framework/flux-core","labels_url":"https://api.github.com/repos/flux-framework/flux-core/issues/614/labels{/name}","comments_url":"https://api.github.com/repos/flux-framework/flux-core/issues/614/comments","events_url":"https://api.github.com/repos/flux-framework/flux-core/issues/614/events","html_url":"https://github.com/flux-framework/flux-core/issues/614","id":143299989,"number":614,"title":"modify lwj KVS usage to slow object store growth","user":{"login":"garlick","id":169947,"avatar_url":"https://avatars.githubusercontent.com/u/169947?v=3","gravatar_id":"","url":"https://api.github.com/users/garlick","html_url":"https://github.com/garlick","followers_url":"https://api.github.com/users/garlick/followers","following_url":"https://api.github.com/users/garlick/following{/other_user}","gists_url":"https://api.github.com/users/garlick/gists{/gist_id}","starred_url":"https://api.github.com/users/garlick/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/garlick/subscriptions","organizations_url":"https://api.github.com/users/garlick/orgs","repos_url":"https://api.github.com/users/garlick/repos","events_url":"https://api.github.com/users/garlick/events{/privacy}","received_events_url":"https://api.github.com/users/garlick/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":{"login":"grondo","id":741970,"avatar_url":"https://avatars.githubusercontent.com/u/741970?v=3","gravatar_id":"","url":"https://api.github.com/users/grondo","html_url":"https://github.com/grondo","followers_url":"https://api.github.com/users/grondo/followers","following_url":"https://api.github.com/users/grondo/following{/other_user}","gists_url":"https://api.github.com/users/grondo/gists{/gist_id}","starred_url":"https://api.github.com/users/grondo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/grondo/subscriptions","organizations_url":"https://api.github.com/users/grondo/orgs","repos_url":"https://api.github.com/users/grondo/repos","events_url":"https://api.github.com/users/grondo/events{/privacy}","received_events_url":"https://api.github.com/users/grondo/received_events","type":"User","site_admin":false},"milestone":null,"comments":45,"created_at":"2016-03-24T16:53:08Z","updated_at":"2016-03-30T16:18:19Z","closed_at":"2016-03-30T16:18:19Z","body":"In #609 the following short term workaround of the way lwj data is stored in the KVS was proposed to check growth of the content-sqlite db file when running many small jobs.\r\n\r\n> The symlink idea seems like a good one.  Since we want the directory containing the running jobs to be the small one (since each job seems to be updated about 16 times, and that's with empty stdio), the symlinks would need to point from the lwj directory to a \"running\" directory while the job is running.   Once the job is completed, the final job state could replace the symlink and be removed from the running directory.\r\n>\r\n> I think this would work transparently for those kvs_watching the lwj._jobid_.state or stdio keys;  watch caches the key and looks it up each time the root ref changes, sending a response to the watcher whenever the value changes.  If there are symlinks in the path that change, the root ref will change and the paths will be walked from the root on each change.\r\n>\r\n> In summary:\r\n* To create a job, create running._jobid_ and symlink lwj._jobid_ to running._jobid_\r\n* Wrexecd updates state in running._jobid_\r\n* Watchers watch lwj._jobid_ as before\r\n* On completion, wrexecd for local rank 0 recursively copies running._jobid_ over lwj._jobid_ and removes running._jobid_."}}