{"action":"created","issue":{"url":"https://api.github.com/repos/dask/dask/issues/3783","repository_url":"https://api.github.com/repos/dask/dask","labels_url":"https://api.github.com/repos/dask/dask/issues/3783/labels{/name}","comments_url":"https://api.github.com/repos/dask/dask/issues/3783/comments","events_url":"https://api.github.com/repos/dask/dask/issues/3783/events","html_url":"https://github.com/dask/dask/issues/3783","id":342709347,"node_id":"MDU6SXNzdWUzNDI3MDkzNDc=","number":3783,"title":"Support Task Annotations within the graph","user":{"login":"sjperkins","id":3530212,"node_id":"MDQ6VXNlcjM1MzAyMTI=","avatar_url":"https://avatars3.githubusercontent.com/u/3530212?v=4","gravatar_id":"","url":"https://api.github.com/users/sjperkins","html_url":"https://github.com/sjperkins","followers_url":"https://api.github.com/users/sjperkins/followers","following_url":"https://api.github.com/users/sjperkins/following{/other_user}","gists_url":"https://api.github.com/users/sjperkins/gists{/gist_id}","starred_url":"https://api.github.com/users/sjperkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sjperkins/subscriptions","organizations_url":"https://api.github.com/users/sjperkins/orgs","repos_url":"https://api.github.com/users/sjperkins/repos","events_url":"https://api.github.com/users/sjperkins/events{/privacy}","received_events_url":"https://api.github.com/users/sjperkins/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":11,"created_at":"2018-07-19T12:34:36Z","updated_at":"2018-08-16T08:53:01Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"I think this issue has some relation to https://github.com/dask/distributed/issues/2127, where @kkraus14 wants to run certain tasks on CPU/GPU workers. I've also wanted to run tasks on specific workers, or require resources to be exclusive for certain tasks.\r\n\r\nCurrently, these task dependencies must be specified as additional arguments to `compute`/`persist` etc. rather than at the point of actual construction -- embedding resource/worker dependencies in the graph is not currently possible.\r\n\r\nTo support this,  how about adding a `TaskAnnotation` type? This can be a namedtuple, itself containing nested tuples representing key-value pairs. e.g.\r\n\r\n```python\r\nannot = TaskAnnotation(an=(('resource', ('GPU': '1'), ('worker', 'alice')))\r\n```\r\n\r\ndask array graphs tend to have the following structure:\r\n\r\n```python\r\ndsk = {\r\n    (tsk_name, 0) : (fn, arg1, arg2, ..., argn),\r\n    (tsk_name, 1) : (fn, arg1, arg2, ..., argn),\r\n}\r\n```\r\n\r\nHow about embedding annotations within value tuples?\r\n\r\n```python\r\ndsk = {\r\n    (tsk_name, 0) : (fn, arg1, arg2, ..., argn, annotation1),\r\n    (tsk_name, 1) : (fn, arg1, arg2, ..., argn, annotation2),\r\n}\r\n```\r\n\r\nIf the scheduler discovers an annotation in the tuple, it could remove it from the argument list and attempt to satisfy the requested constraints. In the above example, annotations are placed at the end of the tuple, but the location could be arbitrary and multiple annotations are possible. Alternatively, it might be better to put them at the start.\r\n\r\nI realise the above example is somewhat specific to dask arrays (I'm not too familiar with the dataframe and bag collections) so there may be issues I'm not seeing.\r\n\r\nOne problem I can immediately identify would be modifying existing graph construction functions to support the above annotations (atop/top support is probably the first place to look)."},"comment":{"url":"https://api.github.com/repos/dask/dask/issues/comments/413473906","html_url":"https://github.com/dask/dask/issues/3783#issuecomment-413473906","issue_url":"https://api.github.com/repos/dask/dask/issues/3783","id":413473906,"node_id":"MDEyOklzc3VlQ29tbWVudDQxMzQ3MzkwNg==","user":{"login":"sjperkins","id":3530212,"node_id":"MDQ6VXNlcjM1MzAyMTI=","avatar_url":"https://avatars3.githubusercontent.com/u/3530212?v=4","gravatar_id":"","url":"https://api.github.com/users/sjperkins","html_url":"https://github.com/sjperkins","followers_url":"https://api.github.com/users/sjperkins/followers","following_url":"https://api.github.com/users/sjperkins/following{/other_user}","gists_url":"https://api.github.com/users/sjperkins/gists{/gist_id}","starred_url":"https://api.github.com/users/sjperkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sjperkins/subscriptions","organizations_url":"https://api.github.com/users/sjperkins/orgs","repos_url":"https://api.github.com/users/sjperkins/repos","events_url":"https://api.github.com/users/sjperkins/events{/privacy}","received_events_url":"https://api.github.com/users/sjperkins/received_events","type":"User","site_admin":false},"created_at":"2018-08-16T08:53:01Z","updated_at":"2018-08-16T08:53:01Z","author_association":"CONTRIBUTOR","body":"I'm putting together a [script](https://github.com/dask/dask/files/2292995/test_task.py.zip) to test out the impact of the `(tuple, annotated_tuple, task)` strategies.\r\n\r\nI'm not reproducing the 10X slower to pickle mentioned by @shoyer in #2299.\r\n\r\nThe pickling sizes are somewhat larger with annotations (but compression reduces this by several factors). \r\n\r\nCurrently, the script produces the following results for 1000 tasks:\r\n\r\n```\r\nTASK CREATION TIMING\r\ntuple 180 nanoseconds\r\nannotated_tuple 462 nanoseconds\r\ntask 804 nanoseconds\r\n\r\nPICKLING TASK TIMING\r\ntuple 2.95 microseconds\r\nannotated_tuple 5.85 microseconds\r\ntask 7.88 microseconds\r\n\r\nPICKLING SIZES\r\ntuple 54753 bytes\r\nannotated_tuple 77828 bytes\r\ntask 77818 bytes\r\n\r\nBZIP2 PICKLED SIZES\r\ntuple 7807 bytes\r\nannotated_tuple 9129 bytes\r\ntask 8909 bytes\r\n```\r\n\r\nI intend to expand it to measure the timing impact on the various schedulers, but will be working on this in an incremental fashion over the next while.\r\n\r\nI also wanted to point out that in the context of https://github.com/dask/dask/issues/3514 that I don't think annotations are an all or nothing affair -- Annotating every single task in the graph would  seems like overkill to me. "}}
{"actor":{"display_login":"sjperkins"}}