title: Simple plots of eigenvectors for sklearn.decomposition.PCA
body: <p>I'm trying to understand how <code>Principal Component Analysis</code> works and I am testing it on the <code>sklearn.datasets.load_iris</code> dataset.  I understand how each step works (e.g. standardize the data, covariance, eigendecomposition, sort for highest eigenvalue, transform original data to new axis using <code>K</code> selected dimensions).</p>
  <p>The next step is to visualize where these <code>eigenvectors</code> are being projected into on the dataset (on the <code>PC1 vs. PC2 plot</code>, right?).</p>
  <h3>Can someone explain how to plot [PC1, PC2, PC3] eigenvectors on a 3D plot of the reduced dimension dataset?</h3>
  <p>Also, am I plotting this 2D version correctly? I'm not sure why my first eigenvector has a shorter length.  Should I multiply by the eigenvalue?</p>
  <hr />
  <p><strong>Here is some of the research I have done to accomplish this:</strong></p>
  <p>The PCA method that I'm following is from :
  <a href="https://plot.ly/ipython-notebooks/principal-component-analysis/#Shortcut---PCA-in-scikit-learn" rel="nofollow noreferrer">https://plot.ly/ipython-notebooks/principal-component-analysis/#Shortcut---PCA-in-scikit-learn</a> (although I don't want to use <code>plotly</code>.  I want to stick with <code>pandas, numpy, sklearn, matplotlib, scipy, and seaborn</code>)</p>
  <p>I've been following this tutorial for plotting eigenvectors it seems pretty simple: <a href="https://stackoverflow.com/questions/18299523/basic-example-for-pca-with-matplotlib">Basic example for PCA with matplotlib</a> but I can't seem to replicate the results with my data.</p>
  <p>I found this but it seems overly complicated for what I'm trying to do and I don't want to have to create a <code>FancyArrowPatch</code>: <a href="https://stackoverflow.com/questions/35508141/plotting-the-eigenvector-of-covariance-matrix-using-matplotlib-and-np-linalg">plotting the eigenvector of covariance matrix using matplotlib and np.linalg</a></p>
  <hr />
  <p><strong>I've tried to make my code as straightforward as possible to follow the other tutorials:</strong></p>
  <pre><code>import numpy as np
  import pandas as pd
  import matplotlib.pyplot as plt
  from sklearn.datasets import load_iris
  from sklearn.preprocessing import StandardScaler
  from sklearn import decomposition
  import seaborn as sns; sns.set_style(&quot;whitegrid&quot;, {'axes.grid' : False})
  
  %matplotlib inline
  np.random.seed(0)
  
  # Iris dataset
  DF_data = pd.DataFrame(load_iris().data, 
                         index = [&quot;iris_%d&quot; % i for i in range(load_iris().data.shape[0])],
                         columns = load_iris().feature_names)
  
  Se_targets = pd.Series(load_iris().target, 
                         index = [&quot;iris_%d&quot; % i for i in range(load_iris().data.shape[0])], 
                         name = &quot;Species&quot;)
  
  # Scaling mean = 0, var = 1
  DF_standard = pd.DataFrame(StandardScaler().fit_transform(DF_data), 
                             index = DF_data.index,
                             columns = DF_data.columns)
  
  # Sklearn for Principal Componenet Analysis
  
  # Dims
  m = DF_standard.shape[1]
  K = 2
  
  # PCA (How I tend to set it up)
  M_PCA = decomposition.PCA(n_components=m)
  DF_PCA = pd.DataFrame(M_PCA.fit_transform(DF_standard), 
                  columns=[&quot;PC%d&quot; % k for k in range(1,m + 1)]).iloc[:,:K]
  
  
  # Plot the eigenvectors
  #https://stackoverflow.com/questions/18299523/basic-example-for-pca-with-matplotlib
  
  # This is where stuff gets weird...
  data = DF_standard
  
  mu = data.mean(axis=0)
  eigenvectors, eigenvalues = M_PCA.components_, M_PCA.explained_variance_ #eigenvectors, eigenvalues, V = np.linalg.svd(data.T, full_matrices=False)
  projected_data = DF_PCA #np.dot(data, eigenvectors)
  
  sigma = projected_data.std(axis=0).mean()
  
  fig, ax = plt.subplots(figsize=(10,10))
  ax.scatter(projected_data[&quot;PC1&quot;], projected_data[&quot;PC2&quot;])
  for axis, color in zip(eigenvectors[:K], [&quot;red&quot;,&quot;green&quot;]):
  #     start, end = mu, mu + sigma * axis ### leads to &quot;ValueError: too many values to unpack (expected 2)&quot;
  
      # So I tried this but I don't think it's correct
      start, end = (mu)[:K], (mu + sigma * axis)[:K] 
      ax.annotate('', xy=end,xytext=start, arrowprops=dict(facecolor=color, width=1.0))
      
  ax.set_aspect('equal')
  plt.show()
  </code></pre>
  <p><a href="https://i.stack.imgur.com/t9st0.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/t9st0.png" alt="enter image description here" /></a></p>