original_docstring:  Calculate Partial correlation which is the degree of association between 
      x and y after removing effect of z. This is done by calculating correlation
      coefficient between the residuals of two linear regressions :
      x\sim z, y\sim z
      See : 1 https://en.wikipedia.org/wiki/Partial_correlation
            2 https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1467-842X.2004.00360.x?casa_token=p_D3joHC8C0AAAAA:qigIZHVfcVi8vsz1j2t7uQYOorrYaF3Tm4lpQOUzqG_J9gJgtFerOyliKBnQPVG187nJxbA-wcbXU3QcOw
            3 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4681537/
            4 http://parker.ad.siu.edu/Olive/slch6.pdf
      :param data : pandas dataframe 
      :param x : Column name in data
      :param y : Column name in data
      :param z : string or list
      :param method : string denoting the correlation type - "pearson" or "spearman"
  
      : returns: a python dictionary with keys as 
          n: Sample size
          r: Partial correlation coefficient
          CI95: 95% parametric confidence intervals
          p-val: p-value
      
docstring: Calculate Partial correlation which is the degree of association between
  x and y after removing effect of z.
docstring_tokens: Calculate Partial correlation which is the degree of association between x and y after removing effect of z .
code: def partial_corr(data=None, x=None, y=None, z=None, method="pearson"):
      """ Calculate Partial correlation which is the degree of association between 
      x and y after removing effect of z. This is done by calculating correlation
      coefficient between the residuals of two linear regressions :
      x\sim z, y\sim z
      See : 1 https://en.wikipedia.org/wiki/Partial_correlation
            2 https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1467-842X.2004.00360.x?casa_token=p_D3joHC8C0AAAAA:qigIZHVfcVi8vsz1j2t7uQYOorrYaF3Tm4lpQOUzqG_J9gJgtFerOyliKBnQPVG187nJxbA-wcbXU3QcOw
            3 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4681537/
            4 http://parker.ad.siu.edu/Olive/slch6.pdf
      :param data : pandas dataframe 
      :param x : Column name in data
      :param y : Column name in data
      :param z : string or list
      :param method : string denoting the correlation type - "pearson" or "spearman"
  
      : returns: a python dictionary with keys as 
          n: Sample size
          r: Partial correlation coefficient
          CI95: 95% parametric confidence intervals
          p-val: p-value
      """
  
      assert data.shape[0] > 2 # Check for atleast 3 samples
      assert x != z # x and z should be distinct
      assert y != z # y and z should be distinct
      assert x != y # x and y should be distinct
      if isinstance(z, list):
          assert x not in z # x and z should be distinct
          assert y not in z # y and z should be distinct
  
      combined_variables = [x,y] # Combine all variables - x, y and z
      for var in z:
          combined_variables.append(var) 
      data = data[combined_variables].dropna() #Drop missing values
      n = data.shape[0] # Number of samples after dropping missing values
      k = data.shape[1] - 2 # Number of covariates
      assert n > 2
  
      if method == "spearman":
          V = data.rank(na_option='keep').cov() # Change data to rank for spearman correlation
      else:
          V = data.astype(float).cov() # Computing Covariance Matrix
      Vi = np.linalg.pinv(V, hermitian=True)   # Computing Inverse Covariance Matrix
      Vi_diag = Vi.diagonal()  # Storing variance
      D = np.diag(np.sqrt(1 / Vi_diag)) # Storing Standard Deviations from diagonal of inverse covariance matrix
      pcor = -1 * (D @ Vi @ D)  
      r = pcor[0,1]
      
      if np.isnan(r):
          return {'n': n, 'r': np.nan, 'CI95%': np.nan, 'p-val': np.nan}
  
      # Finding p-value using student T test
      dof = n - k - 2  # Degree of freedom for multivariate analysis
      tval = r * np.sqrt(dof / (1 - r**2)) # Test statistic
      pval = 2 * t.sf(np.abs(tval), dof) # Calculate p-value corresponding to the test statistic and degree of freedom
  
      ci = compute_ci(r=r, nx=(n - k), ny=(n - k)) #Finding Confidence Interval
      ci=np.round(ci, 3)
      stats = {
          'n': n,
          'r': r,
          'CI95%': [ci],
          'p-val': pval.round(5),
      }
      return stats
code_tokens: def partial_corr ( data = None , x = None , y = None , z = None , method = "pearson" ) : assert data . shape [ 0 ] > 2 assert x != z assert y != z assert x != y if isinstance ( z , list ) : assert x not in z assert y not in z combined_variables = [ x , y ] for var in z : combined_variables . append ( var ) data = data [ combined_variables ] . dropna ( ) n = data . shape [ 0 ] k = data . shape [ 1 ] - 2 assert n > 2 if method == "spearman" : V = data . rank ( na_option = 'keep' ) . cov ( ) else : V = data . astype ( float ) . cov ( ) Vi = np . linalg . pinv ( V , hermitian = True ) Vi_diag = Vi . diagonal ( ) D = np . diag ( np . sqrt ( 1 / Vi_diag ) ) pcor = - 1 * ( D @ Vi @ D ) r = pcor [ 0 , 1 ] if np . isnan ( r ) : return { 'n' : n , 'r' : np . nan , 'CI95%' : np . nan , 'p-val' : np . nan } dof = n - k - 2 tval = r * np . sqrt ( dof / ( 1 - r ** 2 ) ) pval = 2 * t . sf ( np . abs ( tval ) , dof ) ci = compute_ci ( r = r , nx = ( n - k ) , ny = ( n - k ) ) ci = np . round ( ci , 3 ) stats = { 'n' : n , 'r' : r , 'CI95%' : [ ci ] , 'p-val' : pval . round ( 5 ) , } return stats
short_docstring: Calculate Partial correlation which is the degree of association between
  x and y after removing effect of z.
short_docstring_tokens: Calculate Partial correlation which is the degree of association between x and y after removing effect of z .
comment: """ Calculate Partial correlation which is the degree of association between 
      x and y after removing effect of z. This is done by calculating correlation
      coefficient between the residuals of two linear regressions :
      x\sim z, y\sim z
      See : 1 https://en.wikipedia.org/wiki/Partial_correlation
            2 https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1467-842X.2004.00360.x?casa_token=p_D3joHC8C0AAAAA:qigIZHVfcVi8vsz1j2t7uQYOorrYaF3Tm4lpQOUzqG_J9gJgtFerOyliKBnQPVG187nJxbA-wcbXU3QcOw
            3 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4681537/
            4 http://parker.ad.siu.edu/Olive/slch6.pdf
      :param data : pandas dataframe 
      :param x : Column name in data
      :param y : Column name in data
      :param z : string or list
      :param method : string denoting the correlation type - "pearson" or "spearman"
  
      : returns: a python dictionary with keys as 
          n: Sample size
          r: Partial correlation coefficient
          CI95: 95% parametric confidence intervals
          p-val: p-value
      """$SEP$# Check for atleast 3 samples$SEP$# x and z should be distinct$SEP$# y and z should be distinct$SEP$# x and y should be distinct$SEP$# x and z should be distinct$SEP$# y and z should be distinct$SEP$# Combine all variables - x, y and z$SEP$#Drop missing values$SEP$# Number of samples after dropping missing values$SEP$# Number of covariates$SEP$# Change data to rank for spearman correlation$SEP$# Computing Covariance Matrix$SEP$# Computing Inverse Covariance Matrix$SEP$# Storing variance$SEP$# Storing Standard Deviations from diagonal of inverse covariance matrix$SEP$# Finding p-value using student T test$SEP$# Degree of freedom for multivariate analysis$SEP$# Test statistic$SEP$# Calculate p-value corresponding to the test statistic and degree of freedom$SEP$#Finding Confidence Interval