{"action":"created","issue":{"url":"https://api.github.com/repos/ganga-devs/ganga/issues/589","repository_url":"https://api.github.com/repos/ganga-devs/ganga","labels_url":"https://api.github.com/repos/ganga-devs/ganga/issues/589/labels{/name}","comments_url":"https://api.github.com/repos/ganga-devs/ganga/issues/589/comments","events_url":"https://api.github.com/repos/ganga-devs/ganga/issues/589/events","html_url":"https://github.com/ganga-devs/ganga/issues/589","id":161175128,"number":589,"title":"Reduced XML Feature","user":{"login":"rob-c","id":1102021,"avatar_url":"https://avatars.githubusercontent.com/u/1102021?v=3","gravatar_id":"","url":"https://api.github.com/users/rob-c","html_url":"https://github.com/rob-c","followers_url":"https://api.github.com/users/rob-c/followers","following_url":"https://api.github.com/users/rob-c/following{/other_user}","gists_url":"https://api.github.com/users/rob-c/gists{/gist_id}","starred_url":"https://api.github.com/users/rob-c/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rob-c/subscriptions","organizations_url":"https://api.github.com/users/rob-c/orgs","repos_url":"https://api.github.com/users/rob-c/repos","events_url":"https://api.github.com/users/rob-c/events{/privacy}","received_events_url":"https://api.github.com/users/rob-c/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/ganga-devs/ganga/labels/Core","name":"Core","color":"d4c5f9"},{"url":"https://api.github.com/repos/ganga-devs/ganga/labels/enhancement","name":"enhancement","color":"009800"},{"url":"https://api.github.com/repos/ganga-devs/ganga/labels/question","name":"question","color":"fbca04"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-06-20T11:35:33Z","updated_at":"2016-07-01T09:55:37Z","closed_at":null,"body":"Hi all,\r\n\r\nI've been doing some looking at reducing the XML files which we're writing to disk.\r\n\r\nDo we explicitly want to write all default Schema attributes to disk?\r\n\r\nI suppose the answer for the moment is yes we do because we could add a Schema attribute to change the behaviour of the class.\r\n\r\nIf we are careful going forward to not change the Schema in a backwards incompatible way we can explore only writing the objects to disk which we have changed from the default Schema value.\r\nThis makes loading objects from disk much quicker in addition to making the XML sizes on disk much smaller.\r\n\r\nI'm not proposing merging any code to do this but I've played with a private patch to do this in some form and the XML size for a large number of DiracFiles reduces by about 60-70% and the loading of the objects into memory is much quicker.\r\n\r\nDo people have a strong opinion about possibly looking into implementing this?\r\n\r\n(This is already backwards compatible aka, XML with a very reduced schema on disk can be loaded without issue by Ganga already)\r\n\r\nThanks,\r\n\r\nRob"},"comment":{"url":"https://api.github.com/repos/ganga-devs/ganga/issues/comments/229907443","html_url":"https://github.com/ganga-devs/ganga/issues/589#issuecomment-229907443","issue_url":"https://api.github.com/repos/ganga-devs/ganga/issues/589","id":229907443,"user":{"login":"drmarkwslater","id":13677368,"avatar_url":"https://avatars.githubusercontent.com/u/13677368?v=3","gravatar_id":"","url":"https://api.github.com/users/drmarkwslater","html_url":"https://github.com/drmarkwslater","followers_url":"https://api.github.com/users/drmarkwslater/followers","following_url":"https://api.github.com/users/drmarkwslater/following{/other_user}","gists_url":"https://api.github.com/users/drmarkwslater/gists{/gist_id}","starred_url":"https://api.github.com/users/drmarkwslater/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/drmarkwslater/subscriptions","organizations_url":"https://api.github.com/users/drmarkwslater/orgs","repos_url":"https://api.github.com/users/drmarkwslater/repos","events_url":"https://api.github.com/users/drmarkwslater/events{/privacy}","received_events_url":"https://api.github.com/users/drmarkwslater/received_events","type":"User","site_admin":false},"created_at":"2016-07-01T09:55:37Z","updated_at":"2016-07-01T09:55:37Z","body":"I think I agree with @egede here. Though I agree it would be **very** nice to reduce the size of the XML, we would still need to store a snapshot of the defaults used to create that object somewhere at time of creation. Now what you could do is also store any repeated values as one value and refer back to that in the object body though I'm unsure how much improvement that would give. In the 10K `DiracFile` case, this could be very significant I guess though I don't know how many 'repeated' values there are stored in a DIRAC file.\r\n\r\nIf size on disk is the major issue, the we could look at automatic zipping (using the gzip module) as I've suggested before maybe? "}}
{"actor":{"display_login":"drmarkwslater"}}