{"action":"created","issue":{"url":"https://api.github.com/repos/ropensci/drake/issues/440","repository_url":"https://api.github.com/repos/ropensci/drake","labels_url":"https://api.github.com/repos/ropensci/drake/issues/440/labels{/name}","comments_url":"https://api.github.com/repos/ropensci/drake/issues/440/comments","events_url":"https://api.github.com/repos/ropensci/drake/issues/440/events","html_url":"https://github.com/ropensci/drake/issues/440","id":336994665,"node_id":"MDU6SXNzdWUzMzY5OTQ2NjU=","number":440,"title":"A new internal fast-lookup data structure for target metadata","user":{"login":"wlandau","id":1580860,"node_id":"MDQ6VXNlcjE1ODA4NjA=","avatar_url":"https://avatars1.githubusercontent.com/u/1580860?v=4","gravatar_id":"","url":"https://api.github.com/users/wlandau","html_url":"https://github.com/wlandau","followers_url":"https://api.github.com/users/wlandau/followers","following_url":"https://api.github.com/users/wlandau/following{/other_user}","gists_url":"https://api.github.com/users/wlandau/gists{/gist_id}","starred_url":"https://api.github.com/users/wlandau/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wlandau/subscriptions","organizations_url":"https://api.github.com/users/wlandau/orgs","repos_url":"https://api.github.com/users/wlandau/repos","events_url":"https://api.github.com/users/wlandau/events{/privacy}","received_events_url":"https://api.github.com/users/wlandau/received_events","type":"User","site_admin":false},"labels":[{"id":979657452,"node_id":"MDU6TGFiZWw5Nzk2NTc0NTI=","url":"https://api.github.com/repos/ropensci/drake/labels/branch:%20i440%20:wrench:","name":"branch: i440 :wrench:","color":"fef2c0","default":false},{"id":575216142,"node_id":"MDU6TGFiZWw1NzUyMTYxNDI=","url":"https://api.github.com/repos/ropensci/drake/labels/difficulty:%20hard%20:fire:","name":"difficulty: hard :fire:","color":"d4c5f9","default":false},{"id":701210128,"node_id":"MDU6TGFiZWw3MDEyMTAxMjg=","url":"https://api.github.com/repos/ropensci/drake/labels/high%20priority%20:grey_exclamation:","name":"high priority :grey_exclamation:","color":"d93f0b","default":false},{"id":577254680,"node_id":"MDU6TGFiZWw1NzcyNTQ2ODA=","url":"https://api.github.com/repos/ropensci/drake/labels/topic:%20efficiency%20:checkered_flag:","name":"topic: efficiency :checkered_flag:","color":"bfd4f2","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-06-29T13:15:20Z","updated_at":"2018-07-01T20:40:30Z","closed_at":null,"author_association":"COLLABORATOR","body":"### Lessons from #435 and #283:\r\n\r\n- `drake` spends a lot of time repeatedly looking up target-level information it should already know.\r\n- The dependency network is inflexible.\r\n\r\n### Explanation\r\n\r\nI think these and similar problems come from the limitations of `drake`'s data structures. The `plan` is great for users, but lookup is slow, and it contains only indirect information about each target's dependencies. The `graph` has complete dependency information, but it does not have other metadata (`igraph` attributes are a pain), and traversing the whole graph for dependencies is slow.\r\n\r\n### Solution\r\n\r\nI want to put all target-level metadata in a unified fast-lookup data structure. This new `config$meta` will replace individual outputs of `drake_meta()` flying around, and it will decrease reliance on the `plan` and `graph`. For [fast lookup](https://stackoverflow.com/questions/34232865/r-fast-single-item-lookup-from-list-vs-data-table-vs-hash), I want to base this data structure on an environment. For convenience, I want it to have a `storr`-like API. Best of both worlds: a [`storr_environment()` cache](http://richfitz.github.io/storr/reference/storr_environment.html). (Thank you @richfitz for making it available!)\r\n\r\nThere is a lot of refactoring to do, but I think this will make `drake` a whole lot faster and give it a cleaner code base.\r\n"},"comment":{"url":"https://api.github.com/repos/ropensci/drake/issues/comments/401631757","html_url":"https://github.com/ropensci/drake/issues/440#issuecomment-401631757","issue_url":"https://api.github.com/repos/ropensci/drake/issues/440","id":401631757,"node_id":"MDEyOklzc3VlQ29tbWVudDQwMTYzMTc1Nw==","user":{"login":"wlandau","id":1580860,"node_id":"MDQ6VXNlcjE1ODA4NjA=","avatar_url":"https://avatars1.githubusercontent.com/u/1580860?v=4","gravatar_id":"","url":"https://api.github.com/users/wlandau","html_url":"https://github.com/wlandau","followers_url":"https://api.github.com/users/wlandau/followers","following_url":"https://api.github.com/users/wlandau/following{/other_user}","gists_url":"https://api.github.com/users/wlandau/gists{/gist_id}","starred_url":"https://api.github.com/users/wlandau/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wlandau/subscriptions","organizations_url":"https://api.github.com/users/wlandau/orgs","repos_url":"https://api.github.com/users/wlandau/repos","events_url":"https://api.github.com/users/wlandau/events{/privacy}","received_events_url":"https://api.github.com/users/wlandau/received_events","type":"User","site_admin":false},"created_at":"2018-07-01T20:40:30Z","updated_at":"2018-07-01T20:40:30Z","author_association":"COLLABORATOR","body":"There is no barrier, technically. I am glad you checked, especially in https://github.com/richfitz/storr/issues/81. From [these results](https://stackoverflow.com/questions/34232865/r-fast-single-item-lookup-from-list-vs-data-table-vs-hash), plain environments seem to win out in terms of lookup speed, and I just assumed `storr_environment()`s would offer similar speed and more convenience.\r\n\r\nFrom https://github.com/richfitz/storr/issues/81#issuecomment-401631474, I now think the right approach is to have `config$meta` be a plain `new.env()` with all the individual `drake_meta()` lists (augmented with dependency information). Lookup speed, code style, and my ability to solve #435 and #283 will all improve."}}
{"actor":{"display_login":"wlandau"}}