title: What causes the singular matrix error in kernel ridge regression and how to fix it?
body: <p>When building a KTBoost model, I got the following error message:</p>
  <pre><code>1402                             K.flat[::K.shape[0] + 1] += self.alphaReg
  1403                             self.solve_kernel=linalg.inv(K)
  1404  modi = 
  KernelRidge(alpha=self.alphaReg,theta=self.theta,kernel_mat=self.kernel_mat,
  1405      solve_kernel=self.solve_kernel,kernel=self.kernel,n_neighbors=self.n_neighbors
  
  
  /data/anaconda3/lib/python3.7/site-packages/scipy/linalg/basic.py in inv(a, overwrite_a, 
  check_finite)
  977         inv_a, info = getri(lu, piv, lwork=lwork, overwrite_lu=1)
  978     if info &gt; 0:
  979         raise LinAlgError(&quot;singular matrix&quot;)
  980     if info &lt; 0:
  981         raise ValueError('illegal value in %d-th argument of internal '
  
  LinAlgError: singular matrix
  </code></pre>
  <p>I understand that it is usually caused by duplicated or highly correlated variables so excluded such variables before feeding the dataset to the function; however, the problem still happened.</p>
  <p>In addition, adding a small random noise to the dataset did not work; however, if I created a dummy variable which had random numbers between (0, 1) and appended it to the dataset, the problem disappeared and the model ran well.</p>
  <p>Could someone give me an insight on what caused this issue, please?</p>