text: user1369: I wanted to generate a clustering heat map for the microarray data. This is the first time I'm working on Microarray data. I read some tutorials but have few doubts.
  I'm using microarray (Affymetrix SNP 6.0 data) gene expression data. For example the data looks like following:
  ProbeID S1  S2  S3  S4  S5  S6  S7
  10008   131.4311    369.4926    222.0441    687.4181    176.8892    258.1233    316.5573
  10010   78.73022    83.97501    81.56039    86.11443    78.09758    81.88231    84.17101
  10014   90.02816    95.07267    101.1761    93.35585    81.96468    94.3553 93.89527
  10017   79.86837    81.63064    88.19524    79.47265    76.3437 101.6351    93.71674
  10019   99.03493    109.1835    104.97  102.7423    108.3677    98.93459    101.4052
  10020   79.58075    84.28915    90.53562    74.47786    75.96112    96.39649    95.8828
  10021   121.5373    149.9351    146.5956    122.8523    110.5759    132.4268    130.4409
  10025   616.5994    1326.735    1358.187    2315.851    1068.745    3229.759    4435.021
  10035   70.44073    69.56772    68.25446    68.35857    70.86771    74.3843 67.93569
  
  I created expression matrix and did log2 conversion. Now the data looks like following.
              S1  S2  S3  S4  S5  S6  S7
  Gene1   6.429276339 6.338451158 6.333760753 6.419191996 6.503471181 6.329103499 6.211373601
  Gene2   6.379471993 6.296018518 6.237316465 6.2696332   6.329489132 6.359770303 6.240070336
  Gene3   12.84498365 12.00265682 13.92741965 12.553162   13.39001307 13.8933423  12.58695704
  Gene4   8.661860382 6.723004202 6.300975176 7.661012019 7.905219709 6.957578023 6.70945883
  Gene5   6.945382967 6.814979733 6.575916303 6.63460198  6.627380524 6.733926424 6.280618235
  Gene6   9.280222581 8.81560969  9.683073561 9.480038673 8.801438707 9.190943942 7.743705471
  Gene7   6.593564871 6.63502488  6.389962535 6.511360029 6.694572404 6.6492763   6.527199544
  Gene8   6.431615372 6.309515078 6.248153876 6.329288965 6.46768078  6.355268547 6.384284754
  Gene9   7.513349406 7.234654595 7.490935892 6.801368215 7.323811386 7.018733196 7.055932044
  
  Before clustering for normalisation:
  I applied a function for normalisation 
  data = t(apply(data, 1, function(x) {
    q10 = quantile(x, 0.1)
    q90 = quantile(x, 0.9)
    x[x < q10] = q10
    x[x > q90] = q90
    scale(x)
  }))
  
  When I apply the above function I see there are values which are positive and also negative. 
  And when I used "rma" for background correction and "normalise" function from "oligo" package I don't see any negative values.
  I don't understand. 
  Which one should I use to check the clustering among samples? Any help is appreciated.
  
  user1855: Negative values are to be expected as you have log-transformed the data. Any data value below 2 will have a negative log2.
  There is a difference between normalising for analysis and truncating for plotting. The RMA normalisation or similar is intended for differential expression analysis. I would recommend using this normalisation first.
  When plotting, the normalised data may still contain outliers which skew the plotting parameters (such as the colour scale). Due to the large number of genes, there may be some far beyond 3 standard deviations and plotting these at their true value will make it hard to discern trends in the middle. Truncating the data for a heatmap is common, typically at |z| > 3 but quantiles should work much the same. This is a matter of visualisation, not clustering analysis. It would not be needed unless you were plotting the data.
  
  user77: You see negative values with your function because you're setting the average of each row to 0 and its standard deviation to 1.
  In general, I would trust a standard normalization method (rma in this case) more than some random "truncate and then scale the rows" method. Your method isn't even doing any between-array normalization, which is the benefit of rma.
  
original_text: 1369: <p>I wanted to generate a clustering heat map for the microarray data. This is the first time I'm working on Microarray data. I read some tutorials but have few doubts.</p>
  
  <p>I'm using microarray (Affymetrix SNP 6.0 data) gene expression data. For example the data looks like following:</p>
  
  <pre><code>ProbeID S1  S2  S3  S4  S5  S6  S7
  10008   131.4311    369.4926    222.0441    687.4181    176.8892    258.1233    316.5573
  10010   78.73022    83.97501    81.56039    86.11443    78.09758    81.88231    84.17101
  10014   90.02816    95.07267    101.1761    93.35585    81.96468    94.3553 93.89527
  10017   79.86837    81.63064    88.19524    79.47265    76.3437 101.6351    93.71674
  10019   99.03493    109.1835    104.97  102.7423    108.3677    98.93459    101.4052
  10020   79.58075    84.28915    90.53562    74.47786    75.96112    96.39649    95.8828
  10021   121.5373    149.9351    146.5956    122.8523    110.5759    132.4268    130.4409
  10025   616.5994    1326.735    1358.187    2315.851    1068.745    3229.759    4435.021
  10035   70.44073    69.56772    68.25446    68.35857    70.86771    74.3843 67.93569
  </code></pre>
  
  <p>I created expression matrix and did log2 conversion. Now the data looks like following.</p>
  
  <pre><code>            S1  S2  S3  S4  S5  S6  S7
  Gene1   6.429276339 6.338451158 6.333760753 6.419191996 6.503471181 6.329103499 6.211373601
  Gene2   6.379471993 6.296018518 6.237316465 6.2696332   6.329489132 6.359770303 6.240070336
  Gene3   12.84498365 12.00265682 13.92741965 12.553162   13.39001307 13.8933423  12.58695704
  Gene4   8.661860382 6.723004202 6.300975176 7.661012019 7.905219709 6.957578023 6.70945883
  Gene5   6.945382967 6.814979733 6.575916303 6.63460198  6.627380524 6.733926424 6.280618235
  Gene6   9.280222581 8.81560969  9.683073561 9.480038673 8.801438707 9.190943942 7.743705471
  Gene7   6.593564871 6.63502488  6.389962535 6.511360029 6.694572404 6.6492763   6.527199544
  Gene8   6.431615372 6.309515078 6.248153876 6.329288965 6.46768078  6.355268547 6.384284754
  Gene9   7.513349406 7.234654595 7.490935892 6.801368215 7.323811386 7.018733196 7.055932044
  </code></pre>
  
  <p>Before clustering for normalisation:</p>
  
  <p>I applied a function for normalisation </p>
  
  <pre><code>data = t(apply(data, 1, function(x) {
    q10 = quantile(x, 0.1)
    q90 = quantile(x, 0.9)
    x[x &lt; q10] = q10
    x[x &gt; q90] = q90
    scale(x)
  }))
  </code></pre>
  
  <p>When I apply the above function I see there are values which are positive and also negative. </p>
  
  <p>And when I used "rma" for background correction and "normalise" function from "oligo" package I don't see any negative values.</p>
  
  <p>I don't understand. </p>
  
  <p>Which one should I use to check the clustering among samples? Any help is appreciated.</p>
   1855: <p>Negative values are to be expected as you have log-transformed the data. Any data value below 2 will have a negative log2.</p>
  
  <p>There is a difference between normalising for analysis and truncating for plotting. The RMA normalisation or similar is intended for differential expression analysis. I would recommend using this normalisation first.</p>
  
  <p>When plotting, the normalised data may still contain outliers which skew the plotting parameters (such as the colour scale). Due to the large number of genes, there may be some far beyond 3 standard deviations and plotting these at their true value will make it hard to discern trends in the middle. Truncating the data for a heatmap is common, typically at |z| > 3 but quantiles should work much the same. This is a matter of visualisation, not clustering analysis. It would not be needed unless you were plotting the data.</p>
   77: <p>You see negative values with your function because you're setting the average of each row to 0 and its standard deviation to 1.</p>
  
  <p>In general, I would trust a standard normalization method (rma in this case) more than some random "truncate and then scale the rows" method. Your method isn't even doing any between-array normalization, which is the benefit of rma.</p>
  