{"action":"created","issue":{"url":"https://api.github.com/repos/HPI-Information-Systems/Metanome/issues/328","repository_url":"https://api.github.com/repos/HPI-Information-Systems/Metanome","labels_url":"https://api.github.com/repos/HPI-Information-Systems/Metanome/issues/328/labels{/name}","comments_url":"https://api.github.com/repos/HPI-Information-Systems/Metanome/issues/328/comments","events_url":"https://api.github.com/repos/HPI-Information-Systems/Metanome/issues/328/events","html_url":"https://github.com/HPI-Information-Systems/Metanome/issues/328","id":102787942,"number":328,"title":"Reduce the overhead of algorithm executions","user":{"login":"thorsten-papenbrock","id":6872819,"avatar_url":"https://avatars.githubusercontent.com/u/6872819?v=3","gravatar_id":"","url":"https://api.github.com/users/thorsten-papenbrock","html_url":"https://github.com/thorsten-papenbrock","followers_url":"https://api.github.com/users/thorsten-papenbrock/followers","following_url":"https://api.github.com/users/thorsten-papenbrock/following{/other_user}","gists_url":"https://api.github.com/users/thorsten-papenbrock/gists{/gist_id}","starred_url":"https://api.github.com/users/thorsten-papenbrock/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/thorsten-papenbrock/subscriptions","organizations_url":"https://api.github.com/users/thorsten-papenbrock/orgs","repos_url":"https://api.github.com/users/thorsten-papenbrock/repos","events_url":"https://api.github.com/users/thorsten-papenbrock/events{/privacy}","received_events_url":"https://api.github.com/users/thorsten-papenbrock/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/HPI-Information-Systems/Metanome/labels/bug","name":"bug","color":"fc2929"},{"url":"https://api.github.com/repos/HPI-Information-Systems/Metanome/labels/niceToStartWith","name":"niceToStartWith","color":"009800"},{"url":"https://api.github.com/repos/HPI-Information-Systems/Metanome/labels/question","name":"question","color":"cc317c"}],"state":"closed","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/HPI-Information-Systems/Metanome/milestones/14","html_url":"https://github.com/HPI-Information-Systems/Metanome/milestones/March%202016","labels_url":"https://api.github.com/repos/HPI-Information-Systems/Metanome/milestones/14/labels","id":1587935,"number":14,"title":"March 2016","description":"","creator":{"login":"tabergma","id":7203823,"avatar_url":"https://avatars.githubusercontent.com/u/7203823?v=3","gravatar_id":"","url":"https://api.github.com/users/tabergma","html_url":"https://github.com/tabergma","followers_url":"https://api.github.com/users/tabergma/followers","following_url":"https://api.github.com/users/tabergma/following{/other_user}","gists_url":"https://api.github.com/users/tabergma/gists{/gist_id}","starred_url":"https://api.github.com/users/tabergma/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tabergma/subscriptions","organizations_url":"https://api.github.com/users/tabergma/orgs","repos_url":"https://api.github.com/users/tabergma/repos","events_url":"https://api.github.com/users/tabergma/events{/privacy}","received_events_url":"https://api.github.com/users/tabergma/received_events","type":"User","site_admin":false},"open_issues":0,"closed_issues":11,"state":"open","created_at":"2016-02-17T09:32:43Z","updated_at":"2016-03-10T08:54:03Z","due_on":"2016-03-30T22:00:00Z","closed_at":null},"comments":1,"created_at":"2015-08-24T13:03:07Z","updated_at":"2016-03-10T08:54:03Z","closed_at":"2016-03-10T08:54:03Z","body":"Sometimes the overhead of algorithm executions exeeds the runtime of the algorithms by far. Hence, we must identify the reason for this and fix it.\r\n\r\n\"Der HyFD Algorithmus läuft ~40 Sekunden auf dem horse Datensatz:\r\n~5 Sekunden Ausführung des Algorithmus\r\n~15 Sekunden Aufräumungen nach der Algorithmus Ausführung (größtenteils Objekte in Hibernate speichern)\r\n~20 Sekunden Ergebnisse wieder von Platte lesen, um sie anschließend anzeigen zu können\"\r\n\r\nThe reason might be the parsing of results into JSON and back into Objects. If so, we should replace JSON by some more read- and write-efficient (maybe own) result format."},"comment":{"url":"https://api.github.com/repos/HPI-Information-Systems/Metanome/issues/comments/194741039","html_url":"https://github.com/HPI-Information-Systems/Metanome/issues/328#issuecomment-194741039","issue_url":"https://api.github.com/repos/HPI-Information-Systems/Metanome/issues/328","id":194741039,"user":{"login":"tabergma","id":7203823,"avatar_url":"https://avatars.githubusercontent.com/u/7203823?v=3","gravatar_id":"","url":"https://api.github.com/users/tabergma","html_url":"https://github.com/tabergma","followers_url":"https://api.github.com/users/tabergma/followers","following_url":"https://api.github.com/users/tabergma/following{/other_user}","gists_url":"https://api.github.com/users/tabergma/gists{/gist_id}","starred_url":"https://api.github.com/users/tabergma/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tabergma/subscriptions","organizations_url":"https://api.github.com/users/tabergma/orgs","repos_url":"https://api.github.com/users/tabergma/repos","events_url":"https://api.github.com/users/tabergma/events{/privacy}","received_events_url":"https://api.github.com/users/tabergma/received_events","type":"User","site_admin":false},"created_at":"2016-03-10T08:54:03Z","updated_at":"2016-03-10T08:54:03Z","body":"The overhead comes from converting the results into JSON, writing it to disk, reading the JSON again and converting it into a result object again.\r\nTherefore, we implemented a custom way of writing UCCs, INDs, FDs and ODs. This is a lot faster. All of these result types implement the two methods “toString” and “fromString”. Thereby, the columns are decoded by numbers, which are defined in a header.\r\nFor CUCCs and Basic Statistics we did not implement those methods. CUCCs are quite complicated, we would need a grammar for writing them to disk. Because we do not know what kind of columns the basic statistic result has, we also did not implemented for this result type.\r\n\r\nThere is still a little bit overhead by the creation of the new process and passing parameter via hibernate (store and retrieve). But this ‘overhead’ is necessary and can not be reduced."}}