title: Feed CNN features to KNN algorithm in python
body: <p>I want to build an end-to-end trainable model that do:</p>
  
  <ul>
  <li>extract features from face image by using CNN</li>
  <li>apply KNN algorithm on the previously extracted features</li>
  </ul>
  
  <p>I build my cnn network using python3.5</p>
  
  <pre><code>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=4)
  
  X_train = X_train.astype('float32')
  X_test = X_test.astype('float32')
  
  X_train /= 255
  X_test /= 255
  
  Y_train = np_utils.to_categorical(y_train, nb_classes)
  Y_test = np_utils.to_categorical(y_test, nb_classes)    
  model = Sequential()
      X_train = X_train.reshape(X_train.shape[0], 1, img_rows, img_cols)
      X_test = X_test.reshape(X_test.shape[0], 1, img_rows, img_cols)
  
      model.add(Convolution2D(nb_filters, (nb_conv, nb_conv),
                              padding='valid',
                              input_shape=(1, img_cols, img_rows),
                              data_format='channels_first'))
      model.add(Activation('relu'))
      model.add(Convolution2D(nb_filters, (nb_conv, nb_conv)))
      model.add(Activation('relu'))
      model.add(MaxPooling2D(pool_size=(nb_pool, nb_pool)))
      model.add(Dropout(0.5))
      model.add(Flatten())
      model.add(Dense(128))
      model.add(Activation('relu'))
      model.add(Dropout(0.5))
      model.add(Dense(nb_classes))
      model.add(Activation('softmax'))
  
      model.compile(loss='categorical_crossentropy', optimizer='adadelta', metrics=['accuracy'])
  
  
      hist = model.fit(X_train, Y_train, batch_size=batch_size, epochs=nb_epoch,
                       verbose=1, validation_data=(X_test, Y_test))
  
      hist = model.fit(X_train, Y_train, batch_size=batch_size, epochs=nb_epoch,
                       verbose=1, validation_split=0.2)
  </code></pre>
  
  <p>and also i have the knn algorithm from <a href="https://machinelearningmastery.com/tutorial-to-implement-k-nearest-neighbors-in-python-from-scratch/" rel="nofollow noreferrer">https://machinelearningmastery.com/tutorial-to-implement-k-nearest-neighbors-in-python-from-scratch/</a></p>
  
  <pre><code>def loadDataset(filename, split, trainingSet=[], testSet=[]):
      with open(filename, 'r') as csvfile:
          lines = csv.reader(csvfile)
          dataset = list(lines)
          for x in range(len(dataset) - 1):
              for y in range(4):
                  dataset[x][y] = float(dataset[x][y])
              if random.random() &lt; split:
                  trainingSet.append(dataset[x])
              else:
                  testSet.append(dataset[x])
  
  def euclideanDistance(instance1, instance2, length):
      distance = 0
      for x in range(length):
          distance += pow((instance1[x] - instance2[x]), 2)
      return math.sqrt(distance)
  
  def getNeighbors(trainingSet, testInstance, k):
      distances = []
      length = len(testInstance) - 1
      for x in range(len(trainingSet)):
          dist = euclideanDistance(testInstance, trainingSet[x], length)
          distances.append((trainingSet[x], dist))
      distances.sort(key=operator.itemgetter(1))
      neighbors = []
      for x in range(k):
          neighbors.append(distances[x][0])
      return neighbors
  
  def getResponse(neighbors):
      classVotes = {}
      for x in range(len(neighbors)):
          response = neighbors[x][-1]
          if response in classVotes:
              classVotes[response] += 1
          else:
              classVotes[response] = 1
      sortedVotes = sorted(classVotes.items(), key=operator.itemgetter(1), reverse=True)
      return sortedVotes[0][0]
  </code></pre>
  
  <p>I want to use the extracted features from cnn and feed them into the knn algorithm. How can i get, visualize the extracted features and interact with them by taking each k nearest features and classify them in specific group in knn? I cannot able to link the two codes and get the required result. </p>