title: Why does my Scikit-Learn Linear Regression model only work for certain random states?
body: <p>I am trying to run a machine learning linear regression model to predict the review score of different whiskies, using this data set:  <a href="https://whiskyanalysis.com/index.php/database/" rel="nofollow noreferrer">https://whiskyanalysis.com/index.php/database/</a></p>
  
  <p>Where I use the train/test split method I can specify a random state. If I specify certain ramdon states (e.g. random_state = 3) the model behaves as intented and the predictions are within expected ranges (~ 8.0). For certain other random_state numbers (e.g. random_state = 2) the model fails and makes several predictions that are far from the expected values (many powers of 10 wrong!)</p>
  
  <p>I am at a loss to explain this behaviour.</p>
  
  <p>I am running sckit-learn v20.0 in Anaconda Navigator 1.9.2, Python 3.6.5.  </p>
  
  <p>Thanks in advance for any help - here is my code:  </p>
  
  <pre><code>import pandas as pd
  import numpy as np
  
  df = pd.read_csv('whisky_analysis.csv')
  df.head()
  
  #get rid of the spaces!
  df.rename(columns={'Meta Critic':'Score','Super Cluster':'Super_Cluster'}, inplace=True)
  
  # replace special character $ with £ 
  df.Cost = df.Cost.str.replace('$','£')
  
  df.shape
  
  df.isna().sum()
  
  # Drop the 'Whisky', 'STDEV' and # columns
  # Drop 'Cluster' and 'Super_Cluster' due to large numbers of NaN  
  df = df.drop(['Whisky', 'STDEV', '#', 'Super_Cluster', 'Cluster'], axis=1)
  
  # drop any remaining rows with NaNs
  df = df.dropna()
  
  #Use one hot encoding to vectorise the categorical variables - we will use pandas' get_dummies 
  df1 = pd.get_dummies(df, columns = ['Cost', 'Type', 'Class','Country'])
  
  # get target and feature vector/matrix
  y = df1.pop('Score')
  y.shape
  X = df1
  X.shape
  
  from sklearn.model_selection import train_test_split
  X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 2)
  
  from sklearn.linear_model import LinearRegression
  reg = LinearRegression().fit(X_train, y_train)
  preds = reg.predict(X_test)
  
  preds
  
  # evaluating performance
  from sklearn.metrics import mean_absolute_error
  mean_absolute_error(y_test, preds)
  
  from sklearn.metrics import r2_score
  r2_score(y_test, preds)
  </code></pre>