{"action":"created","issue":{"url":"https://api.github.com/repos/datalad/datalad/issues/2908","repository_url":"https://api.github.com/repos/datalad/datalad","labels_url":"https://api.github.com/repos/datalad/datalad/issues/2908/labels{/name}","comments_url":"https://api.github.com/repos/datalad/datalad/issues/2908/comments","events_url":"https://api.github.com/repos/datalad/datalad/issues/2908/events","html_url":"https://github.com/datalad/datalad/issues/2908","id":368060151,"node_id":"MDU6SXNzdWUzNjgwNjAxNTE=","number":2908,"title":"Creating subdatasets linearly slows with number of subdatasets","user":{"login":"mih","id":136479,"node_id":"MDQ6VXNlcjEzNjQ3OQ==","avatar_url":"https://avatars1.githubusercontent.com/u/136479?v=4","gravatar_id":"","url":"https://api.github.com/users/mih","html_url":"https://github.com/mih","followers_url":"https://api.github.com/users/mih/followers","following_url":"https://api.github.com/users/mih/following{/other_user}","gists_url":"https://api.github.com/users/mih/gists{/gist_id}","starred_url":"https://api.github.com/users/mih/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mih/subscriptions","organizations_url":"https://api.github.com/users/mih/orgs","repos_url":"https://api.github.com/users/mih/repos","events_url":"https://api.github.com/users/mih/events{/privacy}","received_events_url":"https://api.github.com/users/mih/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-10-09T06:34:40Z","updated_at":"2018-10-10T06:08:04Z","closed_at":null,"author_association":"MEMBER","body":"Variants of this issue have been reported previously #1874 #2015 (private reports). Here is a demo that demos a particular aspect of this issue. Script to create 500 straight subdatasets of a single superdataset:\r\n\r\n```sh\r\nset -e -u\r\n\r\ndatalad create many\r\ncd many\r\nfor i in $(seq 500); do\r\n\techo $i\r\n\t/usr/bin/time -o ../create_timing.txt --append -f \"%D\\t%K\\t%M\\t%e\\t%S\\t%U\" datalad create -d . $i\r\ndone\r\n```\r\n\r\nPlotting the runtime log of each invocation of `create` we can clearly see a trend were the duration more than doubles from the first to the last run:\r\n\r\n![image](https://user-images.githubusercontent.com/136479/46650727-f07fa000-cb9d-11e8-9eed-7efe8cb0576a.png)\r\n\r\nA likely source is the particular behavior of path annotation (which is already being addressed in https://github.com/datalad/datalad-revolution), but other explanations are possible."},"comment":{"url":"https://api.github.com/repos/datalad/datalad/issues/comments/428449388","html_url":"https://github.com/datalad/datalad/issues/2908#issuecomment-428449388","issue_url":"https://api.github.com/repos/datalad/datalad/issues/2908","id":428449388,"node_id":"MDEyOklzc3VlQ29tbWVudDQyODQ0OTM4OA==","user":{"login":"mih","id":136479,"node_id":"MDQ6VXNlcjEzNjQ3OQ==","avatar_url":"https://avatars1.githubusercontent.com/u/136479?v=4","gravatar_id":"","url":"https://api.github.com/users/mih","html_url":"https://github.com/mih","followers_url":"https://api.github.com/users/mih/followers","following_url":"https://api.github.com/users/mih/following{/other_user}","gists_url":"https://api.github.com/users/mih/gists{/gist_id}","starred_url":"https://api.github.com/users/mih/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mih/subscriptions","organizations_url":"https://api.github.com/users/mih/orgs","repos_url":"https://api.github.com/users/mih/repos","events_url":"https://api.github.com/users/mih/events{/privacy}","received_events_url":"https://api.github.com/users/mih/received_events","type":"User","site_admin":false},"created_at":"2018-10-10T06:08:04Z","updated_at":"2018-10-10T06:08:04Z","author_association":"MEMBER","body":"And just as a sanity check: creating, but no registering with a superdataset really happens at constant time...\r\n\r\n![image](https://user-images.githubusercontent.com/136479/46716256-96e1a900-cc63-11e8-8a72-e1793932ebf0.png)\r\n"}}
{"actor":{"display_login":"mih"}}