title: Neural Net Configuration for sin(x)
body: <p>Hey I was wondering if you could look over a neural net I recently followed on a tutorial and give advice on how to improve it and so on. Currently the NN does pretty well for sin(x) from 0 to 1––it is usually .01 off or something. Specifically, I was wondering if recurrence or depth is the way to go to improve results. Also, I was wondering if there is any benefit in increasing batch size during training. Thanks a ton!</p>
  
  <pre><code>import math
  import random
  import numpy as np
  
  def sigmoid(x):
      sigOfX = (1/(1+np.exp(-x)))
      return sigOfX
  
  def grady(x):
      sigmoidGradyOfX = x*(1-x)
      return sigmoidGradyOfX
  
  np.random.seed(1)
  
  # synapses
  syn0 = 2*np.random.random((2,20))-1
  syn1 = 2*np.random.random((20,1))-1
  
  
  for j in range(100000):
      # sin(x) NN training with batch = 1
      # input data for sin(x)
      np.random.seed(j)
      rands= np.random.random((2,1))
      x  = np.ones((2,2))
      x[:,:-1] = rands
      #x is now a matrix with 2 [ random number, 1], [random number, 1] so that I have a bias node
      #y = real solutions
      y = np.array([[math.sin(x[0,0])],
                   [math.sin(x[1,0])]])
  
      l0 = x
      l1 = sigmoid(np.dot(l0,syn0))
      l2 = sigmoid(np.dot(l1,syn1))
  
      l2_error = y - l2
      if(j%5000)==0:
          print (str(round(x[0,0],3)) + " ==&gt; " + str(round(y[0,0],3)) + " || " + str(round(l2[0,0],3)) + "; E = " + str(np.mean(np.abs(l2_error))))
      l2_delta = l2_error*grady(l2)
  
      l1_error = l2_delta.dot(syn1.T)
  
      l1_delta = l1_error*grady(l1)
  
      # updating weights
      syn1 += l1.T.dot(l2_delta)
      syn0 += l0.T.dot(l1_delta)
  #testing the neural net
  x = np.array(
      [[.1, 1],
       [.2, 1],
       [.3, 1],
       [.4, 1],
       [.5, 1],
       [.6, 1],
       [.7, 1],
       [.8, 1],
       ])
  
  l0 = x
  l1 = sigmoid(np.dot(l0, syn0))
  l2 = sigmoid(np.dot(l1, syn1))
  for z in range(0,len(x)):
      print (str(round(x[z,0],4)) + " gives " + str(round(l2[z,0],4)) + " || " +  str(round(math.sin(x[z,0]),4)))
  </code></pre>