title: Normal Equation Implementation in Python / Numpy
body: <p>I've written some beginner code to calculate the co-efficients of a simple linear model using the normal equation.</p>
  
  <pre><code># Modules
  import numpy as np
  
  # Loading data set
  X, y = np.loadtxt('ex1data3.txt', delimiter=',', unpack=True)
  
  data = np.genfromtxt('ex1data3.txt', delimiter=',')
  
  def normalEquation(X, y):
      m = int(np.size(data[:, 1]))
  
      # This is the feature / parameter (2x2) vector that will
      # contain my minimized values
      theta = []
  
      # I create a bias_vector to add to my newly created X vector
      bias_vector = np.ones((m, 1))
  
      # I need to reshape my original X(m,) vector so that I can
      # manipulate it with my bias_vector; they need to share the same
      # dimensions.
      X = np.reshape(X, (m, 1))
  
      # I combine these two vectors together to get a (m, 2) matrix
      X = np.append(bias_vector, X, axis=1)
  
      # Normal Equation:
      # theta = inv(X^T * X) * X^T * y
  
      # For convenience I create a new, tranposed X matrix
      X_transpose = np.transpose(X)
  
      # Calculating theta
      theta = np.linalg.inv(X_transpose.dot(X))
      theta = theta.dot(X_transpose)
      theta = theta.dot(y)
  
      return theta
  
  p = normalEquation(X, y)
  
  print(p)
  </code></pre>
  
  <p>Using the small data set found here: </p>
  
  <p><a href="http://www.lauradhamilton.com/tutorial-linear-regression-with-octave" rel="nofollow noreferrer">http://www.lauradhamilton.com/tutorial-linear-regression-with-octave</a></p>
  
  <p>I get the co-efficients: <code>[-0.34390603; 0.2124426 ]</code> using the above code instead of: <code>[24.9660; 3.3058]</code>. Could anyone help clarify where I am going wrong? </p>