{"action":"created","issue":{"url":"https://api.github.com/repos/pytorch/pytorch/issues/7391","repository_url":"https://api.github.com/repos/pytorch/pytorch","labels_url":"https://api.github.com/repos/pytorch/pytorch/issues/7391/labels{/name}","comments_url":"https://api.github.com/repos/pytorch/pytorch/issues/7391/comments","events_url":"https://api.github.com/repos/pytorch/pytorch/issues/7391/events","html_url":"https://github.com/pytorch/pytorch/issues/7391","id":321353825,"number":7391,"title":"Difference between dilation=1 and dilation=2 convolution outputs","user":{"login":"mel-2445","id":17346443,"avatar_url":"https://avatars1.githubusercontent.com/u/17346443?v=4","gravatar_id":"","url":"https://api.github.com/users/mel-2445","html_url":"https://github.com/mel-2445","followers_url":"https://api.github.com/users/mel-2445/followers","following_url":"https://api.github.com/users/mel-2445/following{/other_user}","gists_url":"https://api.github.com/users/mel-2445/gists{/gist_id}","starred_url":"https://api.github.com/users/mel-2445/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mel-2445/subscriptions","organizations_url":"https://api.github.com/users/mel-2445/orgs","repos_url":"https://api.github.com/users/mel-2445/repos","events_url":"https://api.github.com/users/mel-2445/events{/privacy}","received_events_url":"https://api.github.com/users/mel-2445/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-05-08T21:29:51Z","updated_at":"2018-05-08T22:03:23Z","closed_at":null,"author_association":"NONE","body":"The output of a dilated convolution and a normal convolution over the same inputs have small differences. \r\n\r\n```python\r\nimport torch\r\nfrom torch.autograd import Variable\r\nfrom torch import nn\r\nfrom copy import deepcopy\r\nimport numpy as np\r\ntorch.manual_seed(42)\r\nnp.set_printoptions(suppress=True)\r\nprint(torch.__version__)\r\n```\r\n```\r\n0.4.0\r\n```\r\n\r\n### comparing two normal, equivalent convolutions\r\n```python \r\nimg = Variable(torch.Tensor(1, 1, 8, 8).uniform_(-1, 1).float()).cuda()\r\nconv1 = nn.Conv2d(1,1, kernel_size=(3,3), padding=(1,1), dilation=(1,1), stride=(1,1)).cuda().float()\r\nconv1d = deepcopy(conv1).cuda().float()\r\ny1, y1d = img.clone(), img.clone()\r\nfor i in range(2):\r\n    y1, y1d = conv1(y1), conv1d(y1d)\r\n    print(y1 == y1d)\r\n```\r\n```\r\ntensor([[[[ 1,  1,  1,  1,  1,  1,  1,  1],\r\n          [ 1,  1,  1,  1,  1,  1,  1,  1],\r\n          [ 1,  1,  1,  1,  1,  1,  1,  1],\r\n          [ 1,  1,  1,  1,  1,  1,  1,  1],\r\n          [ 1,  1,  1,  1,  1,  1,  1,  1],\r\n          [ 1,  1,  1,  1,  1,  1,  1,  1],\r\n          [ 1,  1,  1,  1,  1,  1,  1,  1],\r\n          [ 1,  1,  1,  1,  1,  1,  1,  1]]]], dtype=torch.uint8, device='cuda:0')\r\ntensor([[[[ 1,  1,  1,  1,  1,  1,  1,  1],\r\n          [ 1,  1,  1,  1,  1,  1,  1,  1],\r\n          [ 1,  1,  1,  1,  1,  1,  1,  1],\r\n          [ 1,  1,  1,  1,  1,  1,  1,  1],\r\n          [ 1,  1,  1,  1,  1,  1,  1,  1],\r\n          [ 1,  1,  1,  1,  1,  1,  1,  1],\r\n          [ 1,  1,  1,  1,  1,  1,  1,  1],\r\n          [ 1,  1,  1,  1,  1,  1,  1,  1]]]], dtype=torch.uint8, device='cuda:0')\r\n```\r\n\r\n### comparing a normal convolution and dilated convolution\r\n```python\r\nimg = Variable(torch.Tensor(1, 1, 8, 8).uniform_(-1, 1).float()).cuda()\r\nimg2 = Variable(torch.Tensor(1, 1, 16, 16).fill_(0).float()).cuda()\r\nimg2[:,:,::2,::2] = img \r\n# every other element of img2 is equal to img1, so the convolutions should be equivalent\r\nconv1 = nn.Conv2d(1,1, kernel_size=(3,3), padding=(1,1), dilation=(1,1), stride=(1,1)).cuda().float()\r\nconv1d = deepcopy(conv1).cuda().float(); conv1d.padding = 2; conv1d.dilation = 2;\r\ny1, y1d = img.clone(), img2.clone()\r\nfor i in range(2):\r\n    y1, y1d = conv1(y1), conv1d(y1d)\r\n    print(y1 == y1d[:,:,::2,::2])\r\n    print('\\ny1', y1.data.cpu().numpy(), '\\n\\ny1d', y1d[:,:,::2,::2].data.cpu().numpy())\r\n```\r\n\r\n```\r\ntensor([[[[ 0,  1,  1,  1,  0,  0,  0,  1],\r\n          [ 1,  0,  0,  1,  0,  0,  1,  1],\r\n          [ 0,  1,  1,  0,  0,  0,  0,  0],\r\n          [ 0,  0,  0,  0,  1,  0,  0,  0],\r\n          [ 1,  1,  0,  1,  1,  0,  0,  0],\r\n          [ 1,  0,  0,  0,  0,  0,  0,  0],\r\n          [ 1,  0,  0,  0,  0,  0,  0,  0],\r\n          [ 0,  0,  1,  0,  1,  1,  0,  1]]]], dtype=torch.uint8, device='cuda:0')\r\n\r\ny1 [[[[ 0.18245335  0.5571766   0.24222633  0.51272786 -0.0366137\r\n     0.3292828   0.14285116  0.29193938]\r\n   [ 0.44084847 -0.17684549  0.7880291  -0.2545673   0.6045989\r\n     0.11258447  0.346952    0.40679273]\r\n   [ 0.10152698  0.01561394  0.26529396  0.6287596  -0.27929184\r\n     0.11804515  0.6523923  -0.18412307]\r\n   [ 0.21581548  0.24271396  0.2126391   0.61117566  0.5328184\r\n     0.21701872  0.3293224   0.12286359]\r\n   [-0.00887689  0.58292484  0.21940772 -0.1279889   0.5013106\r\n    -0.00609723  0.33294716 -0.02010348]\r\n   [ 0.05689213  0.01242793  0.43355784  0.14659914  0.314588\r\n     0.46874434  0.46879548  0.21148269]\r\n   [ 0.53590834  0.09665629  0.31223047  0.4688882   0.28192094\r\n    -0.18858299  0.42868307 -0.00345975]\r\n   [ 0.32289878  0.2717015  -0.2853805   0.6719317  -0.02764711\r\n     0.58624995  0.24971467  0.34455782]]]] \r\n\r\ny1d [[[[ 0.18245333  0.5571766   0.24222633  0.51272786 -0.03661373\r\n     0.32928282  0.14285117  0.29193938]\r\n   [ 0.44084847 -0.17684546  0.7880292  -0.2545673   0.604599\r\n     0.1125845   0.346952    0.40679273]\r\n   [ 0.10152699  0.01561394  0.26529396  0.62875956 -0.2792919\r\n     0.11804514  0.65239227 -0.18412304]\r\n   [ 0.21581551  0.24271399  0.21263906  0.6111756   0.5328184\r\n     0.21701871  0.32932237  0.12286362]\r\n   [-0.00887689  0.58292484  0.21940771 -0.1279889   0.5013106\r\n    -0.00609717  0.3329472  -0.02010351]\r\n   [ 0.05689213  0.0124279   0.43355787  0.14659913  0.31458798\r\n     0.46874437  0.46879542  0.21148272]\r\n   [ 0.53590834  0.09665631  0.3122305   0.46888816  0.28192097\r\n    -0.18858305  0.4286831  -0.00345972]\r\n   [ 0.32289875  0.27170148 -0.2853805   0.67193174 -0.02764711\r\n     0.58624995  0.24971466  0.34455782]]]]\r\ntensor([[[[ 0,  0,  1,  1,  0,  0,  1,  1],\r\n          [ 0,  0,  1,  0,  0,  0,  1,  1],\r\n          [ 1,  1,  0,  1,  0,  0,  1,  0],\r\n          [ 1,  1,  0,  1,  0,  0,  0,  0],\r\n          [ 0,  1,  0,  0,  0,  0,  1,  1],\r\n          [ 0,  1,  1,  1,  0,  0,  0,  0],\r\n          [ 1,  0,  0,  0,  0,  0,  1,  0],\r\n          [ 1,  1,  0,  0,  1,  1,  1,  0]]]], dtype=torch.uint8, device='cuda:0')\r\n\r\ny1 [[[[ 0.17818487  0.39573178  0.072411    0.37964892  0.11533344\r\n     0.35839948  0.23062417  0.3106264 ]\r\n   [ 0.49025184 -0.03891519  0.80259824 -0.19594035  0.51979357\r\n     0.1743112   0.29046294  0.21686155]\r\n   [ 0.2921512   0.37853616  0.1753643   0.71462727 -0.0153257\r\n     0.35068917  0.51316094  0.03814   ]\r\n   [ 0.35745192  0.33992928  0.12925372  0.23981535  0.31751347\r\n     0.1739238   0.22110583  0.2644839 ]\r\n   [ 0.14615707  0.46480957  0.336934    0.13905695  0.5714524\r\n     0.22194287  0.41280106  0.18325165]\r\n   [ 0.4103817   0.16532618  0.45699972  0.2682967   0.10284683\r\n     0.3202176   0.2178171   0.20150141]\r\n   [ 0.44478476  0.08203305  0.27396432  0.36069816  0.36893907\r\n     0.18250906  0.50857633  0.18990535]\r\n   [ 0.25605363  0.40243214  0.00707495  0.5524442  -0.06527808\r\n     0.45689863  0.03657404  0.32719827]]]] \r\n\r\ny1d [[[[ 0.17818484  0.39573187  0.072411    0.37964892  0.11533341\r\n     0.3583995   0.23062417  0.3106264 ]\r\n   [ 0.4902519  -0.03891525  0.80259824 -0.19594046  0.5197936\r\n     0.17431116  0.29046294  0.21686155]\r\n   [ 0.2921512   0.37853616  0.17536429  0.71462727 -0.01532567\r\n     0.35068923  0.51316094  0.03814001]\r\n   [ 0.35745192  0.33992928  0.1292537   0.23981535  0.31751353\r\n     0.17392376  0.22110581  0.26448393]\r\n   [ 0.14615709  0.46480957  0.33693397  0.13905697  0.5714523\r\n     0.22194286  0.41280106  0.18325165]\r\n   [ 0.41038173  0.16532618  0.45699972  0.2682967   0.1028468\r\n     0.32021767  0.21781707  0.20150146]\r\n   [ 0.44478476  0.08203304  0.27396438  0.3606981   0.3689391\r\n     0.182509    0.50857633  0.18990533]\r\n   [ 0.25605363  0.40243214  0.00707486  0.55244434 -0.06527808\r\n     0.45689863  0.03657404  0.3271983 ]]]]\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/pytorch/pytorch/issues/comments/387556807","html_url":"https://github.com/pytorch/pytorch/issues/7391#issuecomment-387556807","issue_url":"https://api.github.com/repos/pytorch/pytorch/issues/7391","id":387556807,"user":{"login":"soumith","id":1310570,"avatar_url":"https://avatars0.githubusercontent.com/u/1310570?v=4","gravatar_id":"","url":"https://api.github.com/users/soumith","html_url":"https://github.com/soumith","followers_url":"https://api.github.com/users/soumith/followers","following_url":"https://api.github.com/users/soumith/following{/other_user}","gists_url":"https://api.github.com/users/soumith/gists{/gist_id}","starred_url":"https://api.github.com/users/soumith/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/soumith/subscriptions","organizations_url":"https://api.github.com/users/soumith/orgs","repos_url":"https://api.github.com/users/soumith/repos","events_url":"https://api.github.com/users/soumith/events{/privacy}","received_events_url":"https://api.github.com/users/soumith/received_events","type":"User","site_admin":false},"created_at":"2018-05-08T22:03:23Z","updated_at":"2018-05-08T22:03:23Z","author_association":"OWNER","body":"the small differences are expected. You effectively run into floating point semantics. The \"small differences\" become smaller if you compute things in double precision."}}
{"actor":{"display_login":"soumith"}}