{"action":"created","issue":{"url":"https://api.github.com/repos/pmix/master/issues/241","repository_url":"https://api.github.com/repos/pmix/master","labels_url":"https://api.github.com/repos/pmix/master/issues/241/labels{/name}","comments_url":"https://api.github.com/repos/pmix/master/issues/241/comments","events_url":"https://api.github.com/repos/pmix/master/issues/241/events","html_url":"https://github.com/pmix/master/pull/241","id":197018175,"number":241,"title":"Further reduce memory footprint","user":{"login":"rhc54","id":2440928,"avatar_url":"https://avatars.githubusercontent.com/u/2440928?v=3","gravatar_id":"","url":"https://api.github.com/users/rhc54","html_url":"https://github.com/rhc54","followers_url":"https://api.github.com/users/rhc54/followers","following_url":"https://api.github.com/users/rhc54/following{/other_user}","gists_url":"https://api.github.com/users/rhc54/gists{/gist_id}","starred_url":"https://api.github.com/users/rhc54/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rhc54/subscriptions","organizations_url":"https://api.github.com/users/rhc54/orgs","repos_url":"https://api.github.com/users/rhc54/repos","events_url":"https://api.github.com/users/rhc54/events{/privacy}","received_events_url":"https://api.github.com/users/rhc54/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-12-21T19:51:29Z","updated_at":"2016-12-21T19:54:18Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/pmix/master/pulls/241","html_url":"https://github.com/pmix/master/pull/241","diff_url":"https://github.com/pmix/master/pull/241.diff","patch_url":"https://github.com/pmix/master/pull/241.patch"},"body":"Add the ability to compress/decompress long strings per suggestion by @hjelmn. We can't do it exactly as he suggested as that approach only dealt with pack/unpack of buffers for communication. Our problem is more around the storage size, not the size of our messages. So check the length of strings that are about to be stored, and compress them if larger than a limit. Mark such strings with their decompressed size and a new PMIX_COMPRESSED_STRING data type so we know how to decompress them before delivery.\r\n\r\nAlso, remove the HWLOC support from PMIx. We really don't want to store the actual topology tree as (a) it is quite large on some architecures, and (b) the pointers in that tree require that we store a full copy of the tree in each client process. Instead, we will store the HWLOC topology xml string, properly compressed, and provide that back to the caller when requested. The caller will then have to \"load\" the topology tree from the xml string, free the string, and \"destroy\" the tree when they are done with it.\r\n\r\nFixes #204\r\n\r\nSigned-off-by: Ralph Castain <rhc@open-mpi.org>"},"comment":{"url":"https://api.github.com/repos/pmix/master/issues/comments/268622842","html_url":"https://github.com/pmix/master/pull/241#issuecomment-268622842","issue_url":"https://api.github.com/repos/pmix/master/issues/241","id":268622842,"user":{"login":"rhc54","id":2440928,"avatar_url":"https://avatars.githubusercontent.com/u/2440928?v=3","gravatar_id":"","url":"https://api.github.com/users/rhc54","html_url":"https://github.com/rhc54","followers_url":"https://api.github.com/users/rhc54/followers","following_url":"https://api.github.com/users/rhc54/following{/other_user}","gists_url":"https://api.github.com/users/rhc54/gists{/gist_id}","starred_url":"https://api.github.com/users/rhc54/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rhc54/subscriptions","organizations_url":"https://api.github.com/users/rhc54/orgs","repos_url":"https://api.github.com/users/rhc54/repos","events_url":"https://api.github.com/users/rhc54/events{/privacy}","received_events_url":"https://api.github.com/users/rhc54/received_events","type":"User","site_admin":false},"created_at":"2016-12-21T19:54:18Z","updated_at":"2016-12-21T19:54:18Z","body":"@artpol84 @karasevb @kawashima-fj @jjhursey \r\n\r\nPlease take a look at this change as it materially reduces the memory footprint on OMPI. I don't believe your perf tool will see any difference as I don't think it uses strings of sufficient length to trigger the compression.\r\n\r\nI'll be posting an OMPI PR that includes this change and will provide the reference shortly. You can then test it over there to measure the impact both in memory footprint and in wireup time."}}
{"actor":{"display_login":"rhc54"}}