title: LogisticRegression scikit learn covariate (column) order matters on training
body: <p>For some reason the order of the covariates seems to matter with a <a href="http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html" rel="nofollow noreferrer"><code>LogisticRegression</code></a> classifier in scikit-learn, which seems odd to me. I have 9 covariates and a binary output, and when I change the order of the columns and call <code>fit()</code> and then call <code>predict_proba()</code> the output is different. Toy example below</p>
  
  <pre><code>logit_model = LogisticRegression(C=1e9, tol=1e-15)
  </code></pre>
  
  <p>The following</p>
  
  <pre><code>logit_model.fit(df['column_2','column_1'],df['target'])
  logit_model.predict_proba(df['column_2','column_1'])
  
  array([[ 0.27387109,  0.72612891] ..])
  </code></pre>
  
  <p>Gives a different result to:</p>
  
  <pre><code>logit_model.fit(df['column_1','column_2'],df['target'])
  logit_model.predict_proba(df['column_1','column_2'])
  
  array([[ 0.26117794,  0.73882206], ..])
  </code></pre>
  
  <p>This seems surprising to me but maybe thats just my lack of knowledge about the internals of the algorithm and the fit method. </p>
  
  <p>What am I missing?</p>
  
  <p><strong>EDIT:</strong> Here is the full code and data</p>
  
  <p>data: <a href="https://s3-us-west-2.amazonaws.com/gjt-personal/test_model.csv" rel="nofollow noreferrer">https://s3-us-west-2.amazonaws.com/gjt-personal/test_model.csv</a></p>
  
  <pre><code>import pandas as pd
  from sklearn.linear_model import LogisticRegression
  
  df = pd.read_csv('test_model.csv',index_col=False)
  
  columns1 =['col_1','col_2','col_3','col_4','col_5','col_6','col_7','col_8','col_9']
  columns2 =['col_2','col_1','col_3','col_4','col_5','col_6','col_7','col_8','col_9']
  
  logit_model = LogisticRegression(C=1e9, tol=1e-15)
  
  logit_model.fit(df[columns1],df['target'])
  logit_model.predict_proba(df[columns1])
  
  logit_model.fit(df[columns2],df['target'])
  logit_model.predict_proba(df[columns2])
  </code></pre>
  
  <p>Turns out its something to do with <code>tol=1e-15</code> because this gives a different result.</p>
  
  <pre><code>LogisticRegression(C=1e9, tol=1e-15)
  </code></pre>
  
  <p>But this gives the same result.</p>
  
  <pre><code>LogisticRegression(C=1e9)
  </code></pre>