# Question:

why `ConvTranspose2d` can be seen as the gradient of `Conv2d` with respect to its input? 

# Answer:

see blog article: [Blog Link](https://simonjisu.github.io/datascience/2019/10/27/convtranspose2d.html)

## Reference:
- https://arxiv.org/abs/1603.07285
- https://en.wikipedia.org/wiki/Toeplitz_matrix


```python
import sympy
from sympy import Symbol, MatrixSymbol, Matrix
import numpy as np

def convolution(x, w, K, N):
    res = []
    for i in range(N-K+1):
        for j in range(N-K+1):
            a = sum(Matrix(x)[i:(K+i), j:(K+j)].multiply_elementwise(Matrix(w)))
            res.append(a)
    return Matrix(res).reshape(N-K+1, N-K+1)
```


```python
x_input = MatrixSymbol("x^{(l)}", 4, 4)
x_output = MatrixSymbol("x^{(l+1)}", 2, 2)
w = MatrixSymbol("w", 3, 3)
```


```python
o = convolution(x_input, w, K=3, N=4)
o
```




$\displaystyle \left[\begin{matrix}w_{0, 0} x^{(l)}_{0, 0} + w_{0, 1} x^{(l)}_{0, 1} + w_{0, 2} x^{(l)}_{0, 2} + w_{1, 0} x^{(l)}_{1, 0} + w_{1, 1} x^{(l)}_{1, 1} + w_{1, 2} x^{(l)}_{1, 2} + w_{2, 0} x^{(l)}_{2, 0} + w_{2, 1} x^{(l)}_{2, 1} + w_{2, 2} x^{(l)}_{2, 2} & w_{0, 0} x^{(l)}_{0, 1} + w_{0, 1} x^{(l)}_{0, 2} + w_{0, 2} x^{(l)}_{0, 3} + w_{1, 0} x^{(l)}_{1, 1} + w_{1, 1} x^{(l)}_{1, 2} + w_{1, 2} x^{(l)}_{1, 3} + w_{2, 0} x^{(l)}_{2, 1} + w_{2, 1} x^{(l)}_{2, 2} + w_{2, 2} x^{(l)}_{2, 3}\\w_{0, 0} x^{(l)}_{1, 0} + w_{0, 1} x^{(l)}_{1, 1} + w_{0, 2} x^{(l)}_{1, 2} + w_{1, 0} x^{(l)}_{2, 0} + w_{1, 1} x^{(l)}_{2, 1} + w_{1, 2} x^{(l)}_{2, 2} + w_{2, 0} x^{(l)}_{3, 0} + w_{2, 1} x^{(l)}_{3, 1} + w_{2, 2} x^{(l)}_{3, 2} & w_{0, 0} x^{(l)}_{1, 1} + w_{0, 1} x^{(l)}_{1, 2} + w_{0, 2} x^{(l)}_{1, 3} + w_{1, 0} x^{(l)}_{2, 1} + w_{1, 1} x^{(l)}_{2, 2} + w_{1, 2} x^{(l)}_{2, 3} + w_{2, 0} x^{(l)}_{3, 1} + w_{2, 1} x^{(l)}_{3, 2} + w_{2, 2} x^{(l)}_{3, 3}\end{matrix}\right]$




```python
C = o.reshape(1, 4).diff(Matrix(x_input).reshape(16, 1)).reshape(16, 4)
C
```




$\displaystyle \left[\begin{matrix}w_{0, 0} & 0 & 0 & 0\\w_{0, 1} & w_{0, 0} & 0 & 0\\w_{0, 2} & w_{0, 1} & 0 & 0\\0 & w_{0, 2} & 0 & 0\\w_{1, 0} & 0 & w_{0, 0} & 0\\w_{1, 1} & w_{1, 0} & w_{0, 1} & w_{0, 0}\\w_{1, 2} & w_{1, 1} & w_{0, 2} & w_{0, 1}\\0 & w_{1, 2} & 0 & w_{0, 2}\\w_{2, 0} & 0 & w_{1, 0} & 0\\w_{2, 1} & w_{2, 0} & w_{1, 1} & w_{1, 0}\\w_{2, 2} & w_{2, 1} & w_{1, 2} & w_{1, 1}\\0 & w_{2, 2} & 0 & w_{1, 2}\\0 & 0 & w_{2, 0} & 0\\0 & 0 & w_{2, 1} & w_{2, 0}\\0 & 0 & w_{2, 2} & w_{2, 1}\\0 & 0 & 0 & w_{2, 2}\end{matrix}\right]$




```python
# forward 
(Matrix(x_input).reshape(1, 16) @ C).reshape(2, 2)
```




$\displaystyle \left[\begin{matrix}w_{0, 0} x^{(l)}_{0, 0} + w_{0, 1} x^{(l)}_{0, 1} + w_{0, 2} x^{(l)}_{0, 2} + w_{1, 0} x^{(l)}_{1, 0} + w_{1, 1} x^{(l)}_{1, 1} + w_{1, 2} x^{(l)}_{1, 2} + w_{2, 0} x^{(l)}_{2, 0} + w_{2, 1} x^{(l)}_{2, 1} + w_{2, 2} x^{(l)}_{2, 2} & w_{0, 0} x^{(l)}_{0, 1} + w_{0, 1} x^{(l)}_{0, 2} + w_{0, 2} x^{(l)}_{0, 3} + w_{1, 0} x^{(l)}_{1, 1} + w_{1, 1} x^{(l)}_{1, 2} + w_{1, 2} x^{(l)}_{1, 3} + w_{2, 0} x^{(l)}_{2, 1} + w_{2, 1} x^{(l)}_{2, 2} + w_{2, 2} x^{(l)}_{2, 3}\\w_{0, 0} x^{(l)}_{1, 0} + w_{0, 1} x^{(l)}_{1, 1} + w_{0, 2} x^{(l)}_{1, 2} + w_{1, 0} x^{(l)}_{2, 0} + w_{1, 1} x^{(l)}_{2, 1} + w_{1, 2} x^{(l)}_{2, 2} + w_{2, 0} x^{(l)}_{3, 0} + w_{2, 1} x^{(l)}_{3, 1} + w_{2, 2} x^{(l)}_{3, 2} & w_{0, 0} x^{(l)}_{1, 1} + w_{0, 1} x^{(l)}_{1, 2} + w_{0, 2} x^{(l)}_{1, 3} + w_{1, 0} x^{(l)}_{2, 1} + w_{1, 1} x^{(l)}_{2, 2} + w_{1, 2} x^{(l)}_{2, 3} + w_{2, 0} x^{(l)}_{3, 1} + w_{2, 1} x^{(l)}_{3, 2} + w_{2, 2} x^{(l)}_{3, 3}\end{matrix}\right]$




```python
# backward
(Matrix(x_output).reshape(1, 4) @ C.transpose()).reshape(4, 4)
```




$\displaystyle \left[\begin{matrix}w_{0, 0} x^{(l+1)}_{0, 0} & w_{0, 0} x^{(l+1)}_{0, 1} + w_{0, 1} x^{(l+1)}_{0, 0} & w_{0, 1} x^{(l+1)}_{0, 1} + w_{0, 2} x^{(l+1)}_{0, 0} & w_{0, 2} x^{(l+1)}_{0, 1}\\w_{0, 0} x^{(l+1)}_{1, 0} + w_{1, 0} x^{(l+1)}_{0, 0} & w_{0, 0} x^{(l+1)}_{1, 1} + w_{0, 1} x^{(l+1)}_{1, 0} + w_{1, 0} x^{(l+1)}_{0, 1} + w_{1, 1} x^{(l+1)}_{0, 0} & w_{0, 1} x^{(l+1)}_{1, 1} + w_{0, 2} x^{(l+1)}_{1, 0} + w_{1, 1} x^{(l+1)}_{0, 1} + w_{1, 2} x^{(l+1)}_{0, 0} & w_{0, 2} x^{(l+1)}_{1, 1} + w_{1, 2} x^{(l+1)}_{0, 1}\\w_{1, 0} x^{(l+1)}_{1, 0} + w_{2, 0} x^{(l+1)}_{0, 0} & w_{1, 0} x^{(l+1)}_{1, 1} + w_{1, 1} x^{(l+1)}_{1, 0} + w_{2, 0} x^{(l+1)}_{0, 1} + w_{2, 1} x^{(l+1)}_{0, 0} & w_{1, 1} x^{(l+1)}_{1, 1} + w_{1, 2} x^{(l+1)}_{1, 0} + w_{2, 1} x^{(l+1)}_{0, 1} + w_{2, 2} x^{(l+1)}_{0, 0} & w_{1, 2} x^{(l+1)}_{1, 1} + w_{2, 2} x^{(l+1)}_{0, 1}\\w_{2, 0} x^{(l+1)}_{1, 0} & w_{2, 0} x^{(l+1)}_{1, 1} + w_{2, 1} x^{(l+1)}_{1, 0} & w_{2, 1} x^{(l+1)}_{1, 1} + w_{2, 2} x^{(l+1)}_{1, 0} & w_{2, 2} x^{(l+1)}_{1, 1}\end{matrix}\right]$




```python

```

95f9143969da1bfcbda32e6b0a55c08102975e60
ipynb
Jupyter Notebook
02_VISION/02_Conv2d_and_ConvTranspose2d.ipynb
simonjisu/pytorch_tutorials
e87b1dadf940d0deecaca1db86cc101af55be336
MIT
2018-07-05T13:39:35.000Z
2020-11-18T04:39:18.000Z
02_VISION/02_Conv2d_and_ConvTranspose2d.ipynb
simonjisu/pytorch_tutorials
e87b1dadf940d0deecaca1db86cc101af55be336
MIT
02_VISION/02_Conv2d_and_ConvTranspose2d.ipynb
simonjisu/pytorch_tutorials
e87b1dadf940d0deecaca1db86cc101af55be336
MIT
2019-11-29T19:58:09.000Z
2020-11-03T14:33:37.000Z
Qwen/Qwen-72B
1. YES
2. YES