title: K Nearest Neighbour Classifier - random state for train test split leads to different accuracy scores
body: <p>I'm fairly new to data analysis and machine learning. I've been carrying out some KNN classification analysis on a breast cancer dataset in python's sklearn module. I have the following code which attemps to find the optimal k for classification of a target variable.</p>
  <pre><code>from sklearn.datasets import load_breast_cancer
  from sklearn.model_selection import train_test_split
  from sklearn.neighbors import KNeighborsClassifier
  import matplotlib.pyplot as plt
  
  breast_cancer_data = load_breast_cancer()
  
  training_data, validation_data, training_labels, validation_labels = train_test_split(breast_cancer_data.data, breast_cancer_data.target, test_size = 0.2, random_state = 40)
  results = []
  
  for k in range(1,101):
    classifier = KNeighborsClassifier(n_neighbors = k)
    classifier.fit(training_data, training_labels)
    results.append(classifier.score(validation_data, validation_labels))
  
  k_list = range(1,101)
  plt.plot(k_list, results)
  plt.ylim(0.85,0.99)
  plt.xlabel(&quot;k&quot;)
  plt.ylabel(&quot;Accuracy&quot;)
  plt.title(&quot;Breast Cancer Classifier Accuracy&quot;)
  plt.show()
  </code></pre>
  <p>The code loops through 1 to 100 and generates 100 KNN models with 'k' set to incremental values in the range 1 to 100. The performance of each of those models is saved to a list and a plot is generated showing 'k' on the x-axis and model performance on the y-axis.</p>
  <p>The problem I have is that when I change the random_state parameter when spliting the data into training and testing partitions this results in completely different plots indicating varying model performance for different 'k'values for different dataset partitions.</p>
  <p>For me this makes it difficult to decide which 'k' is optimal as the algorithm performs differently for different 'k's using different random states. Surely this doesn't mean that, for this particular dataset, 'k' is arbitrary? Can anyone help shed some light on this?</p>
  <p>Thanks in anticipation</p>
  <p><img src="https://i.stack.imgur.com/FxfN2.png" alt="Random State = 0" />
  <img src="https://i.stack.imgur.com/Iq3Zo.png" alt="Random State = 40" />
  <img src="https://i.stack.imgur.com/ygHCr.png" alt="Random State = 75" /></p>