title: How can I implement locally connected layer in pure Numpy
body: <p>I would like to build a locally connected weight matrix that represents a locally connected neural network in pure python/numpy without deep learning frameworks like Torch or TensorFlow. </p>
  
  <p><a href="https://i.stack.imgur.com/c1kWH.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/c1kWH.png" alt="enter image description here"></a></p>
  
  <p>The weight matrix is a non-square 2D matrix with the dimension (number_input, number_output). (an autoencoder in my case; input>hidden)</p>
  
  <p>So the function I would like to build, take the matrix dimension and the size of the receptive field (number of local connection) and give the associated weight matrix. I've already create a function like this, but for an input size of 8 and an output size of 4 (and RF = 4) my function output :</p>
  
  <pre><code>[[ 0.91822845 0.          0.          0.        ]
  [-0.24264655 -0.54754138  0.          0.        ]
  [ 0.55617366  0.12832513 -0.28733965  0.        ]
  [ 0.27993286 -0.33150324  0.06994107  0.61184121]
  [ 0.          0.04286912 -0.20974503 -0.37633903]
  [ 0.          0.         -0.10386762  0.33553009]
  [ 0.          0.          0.          0.09562682]
  [ 0.          0.          0.          0.        ]]
  </code></pre>
  
  <p>but I would like :</p>
  
  <pre><code>[[ 0.91822845 0.          0.          0.        ]
  [-0.24264655 -0.54754138  0.          0.        ]
  [ 0.55617366  0.12832513  0.          0.        ]
  [ 0          -0.33150324  0.06994107  0         ]
  [ 0.          0.04286912 -0.20974503  0.        ]
  [ 0.          0.         -0.10386762  0.33553009]
  [ 0.          0.          0.11581854  0.09562682]
  [ 0.          0.          0.          0.03448418]]
  </code></pre>
  
  <p>Here's my python code :</p>
  
  <pre><code>import numpy as np
  
  def local_weight(input_size, output_size, RF):
      input_range = 1.0 / input_size ** (1/2)
      w = np.zeros((input_size, output_size))
      for i in range(0, RF):
          for j in range(0, output_size):
              w[j+i, j] = np.random.normal(loc=0, scale=input_range, size=1)
      return w
  
  print(local_weight(8, 4, 4))
  </code></pre>
  
  <p>I look forward for your response!</p>