title: python sklearn ridge regression normalize
body: <p>I generate a simple linear model in which X (dimension D) variables come from multi-normal with 0 covariance. Only the first 10 variables have true coefficients of 1, the rest have coefficients 0. Hence, theoretically, the ridge regression results should be the true coefficients divided by (1+C), where C is the penalty constant. </p>
  
  <pre><code>import numpy as np
  from sklearn import linear_model
  def generate_data(n):
      d = 100
      w = np.zeros(d)
      for i in range(0,10):
          w[i] = 1.0
  
      trainx = np.random.normal(size=(n,d))
      e = np.random.normal(size=(n))
      trainy = np.dot(trainx, w) + e
  
      return trainx, trainy
  </code></pre>
  
  <p>Then I use: </p>
  
  <pre><code>n = 200
  x,y = generate_data(n)
  regr = linear_model.Ridge(alpha=4,normalize=True)
  regr.fit(x, y)
  print(regr.coef_[0:20])
  </code></pre>
  
  <p>Under normalize = True, I get the first 10 coefficients to be somewhere 20% (i.e. 1/(1+4)) of the true value of 1. When normalize = False, I get the first 10 coefficients to be around 1, which are the same results as a simple linear regression model. Moreover, since I generate the data to be mean = 0 and std = 1, normalize = True shouldn't do anything as the data is already "normalized". Can someone explain to me what is going on here? Thanks! </p>