title: Gradient descent algorithm for logic gates
body: <p>I am trying to understand gradient descent algorithm as a self study, by implementing logic gates (And and OR gate). I have written the algorithm for the same, but there seems to be an error with the code in implementation while giving the inputs in np.array(numpy) , seems to be mismatch in array size. Experts please give your thoughts on this for correcting the error.</p>
  <p>Here is my code</p>
  <pre><code># Importing Libraries
  import numpy as np
  import matplotlib.pyplot as plt
  </code></pre>
  <p><strong>cost function and gradient descent algo</strong></p>
  <pre><code>def mean_squared_error(y_true, y_predicted):
      
      # Calculating the loss or cost
      cost = np.sum((y_true-y_predicted)**2) / len(y_true)
      return cost
  
  # Gradient Descent Function
  # Here iterations, learning_rate, stopping_threshold
  # are hyperparameters that can be tuned
  def gradient_descent(x, y, iterations = 100, learning_rate = 0.0001,
                      stopping_threshold = 1e-6):
      
      # Initializing weight, bias, learning rate and iterations
      current_weight = 0.1
      current_bias = 0.01
      iterations = iterations
      learning_rate = learning_rate
      n = float(len(x))
      
      costs = []
      weights = []
      previous_cost = None
      
      # Estimation of optimal parameters
      for i in range(iterations):
          
          # Making predictions
          y_predicted = (current_weight * x) + current_bias
          
          # Calculationg the current cost
          current_cost = mean_squared_error(y, y_predicted)
  
          # If the change in cost is less than or equal to
          # stopping_threshold we stop the gradient descent
          if previous_cost and abs(previous_cost-current_cost)&lt;=stopping_threshold:
              break
          
          previous_cost = current_cost
  
          costs.append(current_cost)
          weights.append(current_weight)
          
          # Calculating the gradients
          weight_derivative = -(2/n) * sum(x * (y-y_predicted))
          bias_derivative = -(2/n) * sum(y-y_predicted)
          
          # Updating weights and bias
          current_weight = current_weight - (learning_rate * weight_derivative)
          current_bias = current_bias - (learning_rate * bias_derivative)
                  
          # Printing the parameters for each 1000th iteration
          #print(f&quot;Iteration {i+1}: Cost {current_cost}, Weight \
          #{current_weight}, Bias {current_bias}&quot;)
    
    # Visualizing the weights and cost at for all iterations
      plt.figure(figsize = (8,6))
      plt.plot(weights, costs)
      plt.scatter(weights, costs, marker='o', color='red')
      plt.title(&quot;Cost vs Weights&quot;)
      plt.ylabel(&quot;Cost&quot;)
      plt.xlabel(&quot;Weight&quot;)
      plt.show()
      
      return current_weight, current_bias
  </code></pre>
  <p>def main()</p>
  <pre><code>def main():
      
      # Data
      X = np.array([[0,0],[0,1],[1,0],[1,1]])
      Y = np.array([[0],[1],[1],[1]])
  
      # Estimating weight and bias using gradient descent
      estimated_weight, eatimated_bias = gradient_descent(X, Y, iterations=2000)
      #print(f&quot;Estimated Weight: {estimated_weight}\nEstimated Bias: {eatimated_bias}&quot;)
  
      # Making predictions using estimated parameters
      Y_pred = estimated_weight*X + eatimated_bias
  
      # Plotting the regression line
      plt.figure(figsize = (8,6))
      plt.scatter(X, Y, marker='o', color='red')
      plt.plot([min(X), max(X)], [min(Y_pred), max(Y_pred)], color='blue',markerfacecolor='red',
              markersize=10,linestyle='dashed')
      plt.xlabel(&quot;X&quot;)
      plt.ylabel(&quot;Y&quot;)
      plt.show()
  </code></pre>
  <p>main function</p>
  <pre><code>if __name__==&quot;__main__&quot;:
      main()
  </code></pre>
  <p><strong>And the error is :</strong></p>
  <pre><code>/usr/local/lib/python3.7/dist-packages/numpy/core/_asarray.py:136: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
    return array(a, dtype, copy=False, order=order, subok=True)
  
  ---------------------------------------------------------------------------
  
  TypeError                                 Traceback (most recent call last)
  
  TypeError: only size-1 arrays can be converted to Python scalars
  
  
  The above exception was the direct cause of the following exception:
  
  ValueError                                Traceback (most recent call last)
  
  &lt;ipython-input-12-c20cdc2df320&gt; in &lt;module&gt;()
        1 if __name__==&quot;__main__&quot;:
  ----&gt; 2         main()
  
  9 frames
  
  /usr/local/lib/python3.7/dist-packages/numpy/core/_asarray.py in asarray(a, dtype, order)
       81 
       82     &quot;&quot;&quot;
  ---&gt; 83     return array(a, dtype, copy=False, order=order)
       84 
       85 
  
  ValueError: setting an array element with a sequence.
  </code></pre>
  <p>Thanks for you help</p>