{"action":"closed","issue":{"url":"https://api.github.com/repos/neuroscout/neuroscout/issues/642","repository_url":"https://api.github.com/repos/neuroscout/neuroscout","labels_url":"https://api.github.com/repos/neuroscout/neuroscout/issues/642/labels{/name}","comments_url":"https://api.github.com/repos/neuroscout/neuroscout/issues/642/comments","events_url":"https://api.github.com/repos/neuroscout/neuroscout/issues/642/events","html_url":"https://github.com/neuroscout/neuroscout/issues/642","id":484718263,"node_id":"MDU6SXNzdWU0ODQ3MTgyNjM=","number":642,"title":"Redundant store / processing of predictor_events","user":{"login":"adelavega","id":2774448,"node_id":"MDQ6VXNlcjI3NzQ0NDg=","avatar_url":"https://avatars0.githubusercontent.com/u/2774448?v=4","gravatar_id":"","url":"https://api.github.com/users/adelavega","html_url":"https://github.com/adelavega","followers_url":"https://api.github.com/users/adelavega/followers","following_url":"https://api.github.com/users/adelavega/following{/other_user}","gists_url":"https://api.github.com/users/adelavega/gists{/gist_id}","starred_url":"https://api.github.com/users/adelavega/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adelavega/subscriptions","organizations_url":"https://api.github.com/users/adelavega/orgs","repos_url":"https://api.github.com/users/adelavega/repos","events_url":"https://api.github.com/users/adelavega/events{/privacy}","received_events_url":"https://api.github.com/users/adelavega/received_events","type":"User","site_admin":false},"labels":[{"id":605708479,"node_id":"MDU6TGFiZWw2MDU3MDg0Nzk=","url":"https://api.github.com/repos/neuroscout/neuroscout/labels/backend","name":"backend","color":"d4c5f9","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2019-08-23T21:26:08Z","updated_at":"2019-09-04T22:55:25Z","closed_at":"2019-09-04T22:55:25Z","author_association":"COLLABORATOR","body":"Two problems:\r\n1) Database size (10-50GB for predictor event table)\r\n2) Query / Report / Compilation speed\r\n - Mostly compilation of the identical runs across subject\r\n - Querying itself I think is not so slow\r\n\r\nSolutions:\r\n\r\nNeuroscout\r\n- Add an intermediate representation of \"unique\" runs (\"configurations\"). This would be upon ingestion, but would map stimuli to these proto-runs, which are them mapped to real runs.\r\n- Skip adding this intermediate representation and see how slow it is to re-create the current API without the `predictor_event` table. So essentially, ignore this table, and try to recreate all functionality\r\n- If the repetition units are known, or can be computed, only ask pybids to compile certain runs\r\n\r\n\r\nPybids\r\n- API for Sparse Run Variable\r\nInstead of writing out files to speed things up \r\n- More elegant build in version of detecting repetitions\r\n"}}
{"actor":{"display_login":"adelavega"}}