{"action":"created","issue":{"url":"https://api.github.com/repos/nestauk/rhodonite/issues/23","repository_url":"https://api.github.com/repos/nestauk/rhodonite","labels_url":"https://api.github.com/repos/nestauk/rhodonite/issues/23/labels{/name}","comments_url":"https://api.github.com/repos/nestauk/rhodonite/issues/23/comments","events_url":"https://api.github.com/repos/nestauk/rhodonite/issues/23/events","html_url":"https://github.com/nestauk/rhodonite/issues/23","id":379718844,"node_id":"MDU6SXNzdWUzNzk3MTg4NDQ=","number":23,"title":"Improve parent finding algorithm with clustering","user":{"login":"georgerichardson","id":6348211,"node_id":"MDQ6VXNlcjYzNDgyMTE=","avatar_url":"https://avatars2.githubusercontent.com/u/6348211?v=4","gravatar_id":"","url":"https://api.github.com/users/georgerichardson","html_url":"https://github.com/georgerichardson","followers_url":"https://api.github.com/users/georgerichardson/followers","following_url":"https://api.github.com/users/georgerichardson/following{/other_user}","gists_url":"https://api.github.com/users/georgerichardson/gists{/gist_id}","starred_url":"https://api.github.com/users/georgerichardson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/georgerichardson/subscriptions","organizations_url":"https://api.github.com/users/georgerichardson/orgs","repos_url":"https://api.github.com/users/georgerichardson/repos","events_url":"https://api.github.com/users/georgerichardson/events{/privacy}","received_events_url":"https://api.github.com/users/georgerichardson/received_events","type":"User","site_admin":false},"labels":[{"id":1029571055,"node_id":"MDU6TGFiZWwxMDI5NTcxMDU1","url":"https://api.github.com/repos/nestauk/rhodonite/labels/enhancement","name":"enhancement","color":"a2eeef","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-11-12T10:45:32Z","updated_at":"2018-11-19T16:21:49Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"Current algorithm:\r\n\r\n1. For communities at time period T, find most similar communities at T-1.\r\n2. Find unions of the possible parent communities up to size n.\r\n3. Find most similar community or union of communities, and label as parents.\r\n\r\nWith clustering:\r\n\r\n1. For each time period, find all combinations of communities up to size n.\r\n2. Cluster the communities and their combinations (hierarchically or OmniSlash)?\r\n3. For a community at time period T, find a pool of potential parents in its community and those nearby.\r\n4. Find most similar community or union of communities, and label as parents.\r\n\r\nPotential advantages:\r\n\r\n- Speed. Currently combinations of potential parent communities are calculated for each child community. This would calculate all combinations once in advance.\r\n- Accuracy. The current algorithm only allows communities that are above a similarity threshold to be considered as candidates in the parent pool. This means that we are excluding the possibility that there are potential parents which are dissimilar to the child, until combined with other potential parents.\r\n\r\nPotential disadvantages:\r\n\r\n- Memory. Will need to compute combinations and cluster on large matrix of potential parents.\r\n- Speed. Single calculation of combinations for all communities and clustering them may take a long time."},"comment":{"url":"https://api.github.com/repos/nestauk/rhodonite/issues/comments/439952035","html_url":"https://github.com/nestauk/rhodonite/issues/23#issuecomment-439952035","issue_url":"https://api.github.com/repos/nestauk/rhodonite/issues/23","id":439952035,"node_id":"MDEyOklzc3VlQ29tbWVudDQzOTk1MjAzNQ==","user":{"login":"georgerichardson","id":6348211,"node_id":"MDQ6VXNlcjYzNDgyMTE=","avatar_url":"https://avatars2.githubusercontent.com/u/6348211?v=4","gravatar_id":"","url":"https://api.github.com/users/georgerichardson","html_url":"https://github.com/georgerichardson","followers_url":"https://api.github.com/users/georgerichardson/followers","following_url":"https://api.github.com/users/georgerichardson/following{/other_user}","gists_url":"https://api.github.com/users/georgerichardson/gists{/gist_id}","starred_url":"https://api.github.com/users/georgerichardson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/georgerichardson/subscriptions","organizations_url":"https://api.github.com/users/georgerichardson/orgs","repos_url":"https://api.github.com/users/georgerichardson/repos","events_url":"https://api.github.com/users/georgerichardson/events{/privacy}","received_events_url":"https://api.github.com/users/georgerichardson/received_events","type":"User","site_admin":false},"created_at":"2018-11-19T16:21:49Z","updated_at":"2018-11-19T16:21:49Z","author_association":"CONTRIBUTOR","body":"Turns out this might be unfeasible as the number of combinations for each year could number in the millions, making the clustering slower than the overall running time of the algorithm.\r\n\r\nA potential better option:\r\n\r\n```\r\nfor clique_future in cliques_future:\r\n    for element in clique_future:\r\n        retrieve cliques from cliques_past that contains element\r\n        find clique(s) that has closest similarity to clique_future\r\n        append to list\r\n    generate combinations of cliques\r\n    find most likely parent\r\n```\r\n\r\nThis should improve accuracy while drastically reducing the problem space for calculating the Jaccard similarity."}}
{"actor":{"display_login":"georgerichardson"}}