{"action":"closed","issue":{"url":"https://api.github.com/repos/pytorch/pytorch/issues/7983","repository_url":"https://api.github.com/repos/pytorch/pytorch","labels_url":"https://api.github.com/repos/pytorch/pytorch/issues/7983/labels{/name}","comments_url":"https://api.github.com/repos/pytorch/pytorch/issues/7983/comments","events_url":"https://api.github.com/repos/pytorch/pytorch/issues/7983/events","html_url":"https://github.com/pytorch/pytorch/issues/7983","id":328016091,"node_id":"MDU6SXNzdWUzMjgwMTYwOTE=","number":7983,"title":"[Feature request] In-place versions of some functions, such as cat or padding","user":{"login":"Randl","id":3028543,"node_id":"MDQ6VXNlcjMwMjg1NDM=","avatar_url":"https://avatars0.githubusercontent.com/u/3028543?v=4","gravatar_id":"","url":"https://api.github.com/users/Randl","html_url":"https://github.com/Randl","followers_url":"https://api.github.com/users/Randl/followers","following_url":"https://api.github.com/users/Randl/following{/other_user}","gists_url":"https://api.github.com/users/Randl/gists{/gist_id}","starred_url":"https://api.github.com/users/Randl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Randl/subscriptions","organizations_url":"https://api.github.com/users/Randl/orgs","repos_url":"https://api.github.com/users/Randl/repos","events_url":"https://api.github.com/users/Randl/events{/privacy}","received_events_url":"https://api.github.com/users/Randl/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-05-31T06:57:29Z","updated_at":"2018-06-13T03:59:57Z","closed_at":"2018-06-13T03:59:56Z","author_association":"NONE","body":"I propose to introduce in-place versions of some functions, which I believe are often used in a way that can benefit from it. In my case those are `cat` and padding but there are probably more (for example https://github.com/pytorch/pytorch/issues/1641). \r\n\r\nThe idea is to make input of the function to share storage with output, for example:\r\n```python3\r\nc = torch.cat_((a,b)) # a is now c[:a_size] and b is c[a_size:]\r\n```\r\nThis wouldn't work in all cases, obviously. If there are tensors sharing storage with `a` or `b`, they will be broken. However, I think often we just concat two outputs to pass them forward or pad input to work with padded one, in which cases that would allow to reduce memory usage significantly.\r\nConsider the following example:\r\n```python3\r\npadder = nn.ReflectionPad2d(1)\r\nconv = nn.Conv2d(input_filters, out_filters, kernel_size=3)\r\n\r\npadded_input = padder(inp) \r\nconv_out = F.tanh(conv(padded_input))\r\noutput = torch.cat(conv_out, inp)\r\n```\r\nIn this case there is no reason to keep separate storage for each of `input`, `padded_input`, and `conv_out` while all of them are actually stored inside the `output`.\r\n\r\nThis optimizations can probably be performed manually, but this is much less convinient and adds unnecessary complexity to the code  "}}
{"actor":{"display_login":"soumith"}}