{"action":"created","issue":{"url":"https://api.github.com/repos/apple/swift/issues/20505","repository_url":"https://api.github.com/repos/apple/swift","labels_url":"https://api.github.com/repos/apple/swift/issues/20505/labels{/name}","comments_url":"https://api.github.com/repos/apple/swift/issues/20505/comments","events_url":"https://api.github.com/repos/apple/swift/issues/20505/events","html_url":"https://github.com/apple/swift/pull/20505","id":379592083,"node_id":"MDExOlB1bGxSZXF1ZXN0MjI5OTk3NjI1","number":20505,"title":"[AutoDiff] Introduce `autodiff_function` instruction","user":{"login":"rxwei","id":856507,"node_id":"MDQ6VXNlcjg1NjUwNw==","avatar_url":"https://avatars0.githubusercontent.com/u/856507?v=4","gravatar_id":"","url":"https://api.github.com/users/rxwei","html_url":"https://github.com/rxwei","followers_url":"https://api.github.com/users/rxwei/followers","following_url":"https://api.github.com/users/rxwei/following{/other_user}","gists_url":"https://api.github.com/users/rxwei/gists{/gist_id}","starred_url":"https://api.github.com/users/rxwei/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rxwei/subscriptions","organizations_url":"https://api.github.com/users/rxwei/orgs","repos_url":"https://api.github.com/users/rxwei/repos","events_url":"https://api.github.com/users/rxwei/events{/privacy}","received_events_url":"https://api.github.com/users/rxwei/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-11-12T01:53:59Z","updated_at":"2018-11-12T05:04:35Z","closed_at":null,"author_association":"COLLABORATOR","pull_request":{"url":"https://api.github.com/repos/apple/swift/pulls/20505","html_url":"https://github.com/apple/swift/pull/20505","diff_url":"https://github.com/apple/swift/pull/20505.diff","patch_url":"https://github.com/apple/swift/pull/20505.patch"},"body":"### Introduction\r\n\r\nAn `autodiff_function` instruction bundles a function operand with its associated differentiation functions up to a specified differentiation order into an ``@autodiff`` function. Normally, there are 4 associated functions per differentiation order: a Jacobian-vector products (JVP) function, a thick differential function that acts as the JVP function's result, vector-Jacobian products (VJP) function, and a thick pullback function that acts as the VJP function's result. When ``[legacy_reverse]`` is specified, only first-order reverse-mode differentiation is legal, and only two functions are being bundled: the primal function and the adjoint function.\r\n\r\nThis instruction will be triggered by an `AutoDiffFunctionExpr` (not yet introduced) that converts a non-differentiable function to an `@autodiff` function. Here is an example of when that happens:\r\n```swift\r\nfunc bar(_: Float) -> Float { ... }\r\n\r\nfunc diff(_ f: @autodiff (Float) -> Float) {\r\n  _ = gradient(of: f)\r\n}\r\n\r\ndiff(bar) // (apply_expr (autodiff_function_expr \"diff\" args=(\"bar\")))\r\n```\r\n\r\n### Syntax\r\n```bnf\r\nsil-instruction ::= 'autodiff_function'\r\n                    sil-autodiff-function-legacy-reverse-mode?\r\n                    sil-autodiff-function-parameter-indices?\r\n                    sil-autodiff-function-order?\r\n                    sil-value ':' sil-type\r\n                    sil-autodiff-associated-functions-clause?\r\n                      \r\nsil-autodiff-function-legacy-reverse-mode ::= '[' 'legacy_reverse' ']'\r\nsil-autodiff-function-parameter-indices ::= '[' 'wrt' [0-9]+ (',', [0-9]+)* ']'\r\nsil-autodiff-function-order ::= '[' 'order' [0-9]+ ']'\r\nsil-autodiff-associated-functions-clause ::= 'with' sil-autodiff-associated-function-list\r\n                                             (',' sil-autodiff-associated-function-list)*\r\nsil-autodiff-associated-function-list ::= '{' sil-value (',' sil-value)* '}'\r\n```\r\n\r\n### Examples\r\n```swift\r\n  autodiff_function [wrt 0] [order 1] %0 : $(T) -> T\r\n      with {%1 : $(T) -> (T) -> T, %2 : $(T, @box {...}) -> T, %3 : $(T) -> (T) -> T, %4 : $(T, @box {...}) -> T}\r\n\r\n  autodiff_function [legacy_reverse] [wrt 0] [order 1] %0 : $(T) -> T\r\n      with {%1 : $(T) -> (..., T), %2 : $(T, ..., T, T) -> T}\r\n```\r\n\r\n### Semantics\r\n\r\n``[parameters ...]`` specifies parameter indices that the original function is differentiable with respect to. When not specified, it defaults to all parameters.\r\n\r\n``[order ...]`` specifies the maximum differentiation order for the resulting function. The number of lists of associated functions is equal to the order.\r\n\r\nA ``with`` clause specifies the differentiation functions associated with the original function. When a ``with`` clause is not specified, the first operand will be differentiated to produce associated functions, and a ``with`` clause will be added to the instruction.\r\n\r\n\r\n### Bundle Layout\r\n\r\nThe layout of an `@autodiff` function starts with the original function, followed by parameter indices, the differentiation order, and a flat list of associated functions.\r\n\r\n1. Standard mode AD. The implementation of the future design described in [the manifesto](https://gist.github.com/rxwei/30ba75ce092ab3b0dce4bde1fc2c9f1d) will use this semantics. \r\n    ```\r\n                   Order 1                  Order 2                 ...\r\n    |----------| |-----|----|-----|----| |-----|----|-----|----| ...\r\n    | Original | | JVP | DF | VJP | PB | | JVP | DF | VJP | PB | ...\r\n    |----------| |-----|----|-----|----| |-----|----|-----|----| ...\r\n    ```\r\n    Note: \"DF\" means differential. \"PB\" means pullback.\r\n\r\n2. Legacy (existing) reverse mode AD. This is to support today's reverse-mode AD and will be deprecated when the standard mode is implemented.\r\n    ```\r\n                   Order 1\r\n    |----------| |--------|---------|\r\n    | Original | | Primal | Adjoint |\r\n    |----------| |--------|---------|\r\n    ```"},"comment":{"url":"https://api.github.com/repos/apple/swift/issues/comments/437757548","html_url":"https://github.com/apple/swift/pull/20505#issuecomment-437757548","issue_url":"https://api.github.com/repos/apple/swift/issues/20505","id":437757548,"node_id":"MDEyOklzc3VlQ29tbWVudDQzNzc1NzU0OA==","user":{"login":"rxwei","id":856507,"node_id":"MDQ6VXNlcjg1NjUwNw==","avatar_url":"https://avatars0.githubusercontent.com/u/856507?v=4","gravatar_id":"","url":"https://api.github.com/users/rxwei","html_url":"https://github.com/rxwei","followers_url":"https://api.github.com/users/rxwei/followers","following_url":"https://api.github.com/users/rxwei/following{/other_user}","gists_url":"https://api.github.com/users/rxwei/gists{/gist_id}","starred_url":"https://api.github.com/users/rxwei/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rxwei/subscriptions","organizations_url":"https://api.github.com/users/rxwei/orgs","repos_url":"https://api.github.com/users/rxwei/repos","events_url":"https://api.github.com/users/rxwei/events{/privacy}","received_events_url":"https://api.github.com/users/rxwei/received_events","type":"User","site_admin":false},"created_at":"2018-11-12T05:04:35Z","updated_at":"2018-11-12T05:04:35Z","author_association":"COLLABORATOR","body":"@swift-ci please test tensorflow"}}
{"actor":{"display_login":"rxwei"}}