dna microarray analysis has gained widespread application and a wide variety of methods have been developed to analyze the large and complex datasets generated by this technology ( 1 ) . due to the sheer volume of data , and the high number of sources of potential error , quality control and quality assurance
since error may be the result of many factors at multiple steps , a number of qc / qa measures have been proposed to monitor specific sources of error .
most of these methods focus on individual spots or spots within a single array ( 3 ) .
other methods monitor an individual array as a whole during the experimental process using classification methods ( 4 ) or by comparison of the statistical features of an array with the same statistical features based on historical data ( 5 ) . here
we report an r function ( www.rproject.org ) named microarray outlier filter ( mof ) , which was designed to screen outliers at the whole array level by using the arrays from the current experiment and those from the historical archive that meet defined criteria .
our lab now routinely applies this software to the qa of our microarray experiments ( 6 ) .
in essence , mof examines the consistency of arrays in a large scale experiment ( multiple arrays ) .
arrays , as a whole , are inspected to reveal those arrays that are obviously different from most others .
these few  abnormal  arrays are most likely failed arrays that may contain unreliable data .
this analysis is based on two assumptions : first , that all samples are similar in gene expression profile ( that is , technical or biological replicates ) and that they have been subjected to the same experimental procedure ( in principle , the results should be similar between arrays since expression levels should be uniform for the majority of the probes ) ; second , that data from most of the arrays are of good quality , resulting in only a few unusual arrays being labeled as potentially failed ones .
otherwise , it is possible for a systematic experimental error to result in the few good arrays being suspected as failures because they are identified as  abnormal  by mof .
note that historical arrays satisfying these two criteria can be included to generate a larger dataset for analysis of the outliers .
an outlier data point is defined in the context of all the data points for the same specific probe , across all the arrays .
the resistant z - score is used to tag an outlier data point , which is defined as : zi = xixswhere x and s are the median and median absolute deviation values , respectively , for each probe across all arrays .
a data point is designated as an outlier if its resistant z - score falls outside a preset threshold , which may be 3 , 4 or 5 in our experience .
the percentage of outliers among all data points in consideration from an array is used as an indicator of the quality of the array .
this is based on our observation that an outlier array , which is largely different from the majority of comparable arrays in its gene expression profile , tends to have more data points distributed at extremes .
the other statistical index is the pearson correlation coefficient , expressed as : r=i=1k(xix)(yiy)(n1)sxsywhere x and y are two variables of the signal intensities from the two arrays , n is the number of probes included , x and y stand for means , sx and sy stand for standard deviations of two variables , respectively .
the correlation coefficient between two arrays is computed by a function provided in the r package .
each array is represented by a collection of data points in the same order of probes .
thus , the correlation coefficient reflects the similarity of the two arrays in regard to the expression levels for all the probes collectively .
if an array displays apparently low correlation or even reverse correlation to many other arrays , it is flagged as an unusual array merit closer review for failure .
therefore , mof can be used to assist in the detection of potentially failed arrays . as an r function
, mof is a ready - to - use tool for listing the arrays by the possibility of failure .
it does not mean that there must be failed arrays in each batch of arrays .
thus it is very important for the user to further verify that the outlier arrays are actually failed arrays by other means , such as scatter plots , clustering , etc .
, starting from the worst outlier arrays . in our experience , the two lists determined by the mof indices often show the same  unusual  arrays , thus confirming each other .
however , since these two indices do not reflect exactly the same properties of the data , they may detect different abnormalities in the arrays and thus complement each other .
again , additional validation is critical to identify the truly failed arrays and the underlying causes .
the input text file is a matrix containing preprocessed microarray data with rows as probes and columns as arrays . for maximum reliability ,
data points with expression levels close to the background or within the scanner saturation range should be discarded .
the output is three text files and two heat maps . in the first text file ,
two lists of arrays are ordered according to the average correlation coefficient to the rest of the arrays and percentage of outlier spots , respectively .
a correlation coefficient table containing the correlation coefficients for all pairs of arrays is found in the second text file .
a heat map is generated to provide a general visualization of this table ( figure 1a ) as a guide for the utilization of the data for additional detailed analysis .
similarly , percentages of outlier spots for all the arrays are also given as a table in a text file and a corresponding heat map ( figure 1b ) .
in essence , mof examines the consistency of arrays in a large scale experiment ( multiple arrays ) .
arrays , as a whole , are inspected to reveal those arrays that are obviously different from most others .
these few  abnormal  arrays are most likely failed arrays that may contain unreliable data .
this analysis is based on two assumptions : first , that all samples are similar in gene expression profile ( that is , technical or biological replicates ) and that they have been subjected to the same experimental procedure ( in principle , the results should be similar between arrays since expression levels should be uniform for the majority of the probes ) ; second , that data from most of the arrays are of good quality , resulting in only a few unusual arrays being labeled as potentially failed ones .
otherwise , it is possible for a systematic experimental error to result in the few good arrays being suspected as failures because they are identified as  abnormal  by mof .
note that historical arrays satisfying these two criteria can be included to generate a larger dataset for analysis of the outliers .
an outlier data point is defined in the context of all the data points for the same specific probe , across all the arrays .
the resistant z - score is used to tag an outlier data point , which is defined as : zi = xixswhere x and s are the median and median absolute deviation values , respectively , for each probe across all arrays .
a data point is designated as an outlier if its resistant z - score falls outside a preset threshold , which may be 3 , 4 or 5 in our experience .
the percentage of outliers among all data points in consideration from an array is used as an indicator of the quality of the array .
this is based on our observation that an outlier array , which is largely different from the majority of comparable arrays in its gene expression profile , tends to have more data points distributed at extremes .
the other statistical index is the pearson correlation coefficient , expressed as : r=i=1k(xix)(yiy)(n1)sxsywhere x and y are two variables of the signal intensities from the two arrays , n is the number of probes included , x and y stand for means , sx and sy stand for standard deviations of two variables , respectively .
the correlation coefficient between two arrays is computed by a function provided in the r package .
each array is represented by a collection of data points in the same order of probes .
thus , the correlation coefficient reflects the similarity of the two arrays in regard to the expression levels for all the probes collectively .
if an array displays apparently low correlation or even reverse correlation to many other arrays , it is flagged as an unusual array merit closer review for failure .
therefore , mof can be used to assist in the detection of potentially failed arrays . as an r function
, mof is a ready - to - use tool for listing the arrays by the possibility of failure .
it does not mean that there must be failed arrays in each batch of arrays .
thus it is very important for the user to further verify that the outlier arrays are actually failed arrays by other means , such as scatter plots , clustering , etc .
, starting from the worst outlier arrays . in our experience , the two lists determined by the mof indices often show the same  unusual  arrays , thus confirming each other .
however , since these two indices do not reflect exactly the same properties of the data , they may detect different abnormalities in the arrays and thus complement each other .
again , additional validation is critical to identify the truly failed arrays and the underlying causes .
the input text file is a matrix containing preprocessed microarray data with rows as probes and columns as arrays . for maximum reliability ,
data points with expression levels close to the background or within the scanner saturation range should be discarded .
the output is three text files and two heat maps . in the first text file ,
two lists of arrays are ordered according to the average correlation coefficient to the rest of the arrays and percentage of outlier spots , respectively .
a correlation coefficient table containing the correlation coefficients for all pairs of arrays is found in the second text file .
a heat map is generated to provide a general visualization of this table ( figure 1a ) as a guide for the utilization of the data for additional detailed analysis .
similarly , percentages of outlier spots for all the arrays are also given as a table in a text file and a corresponding heat map ( figure 1b ) .
using mof requires setting proper thresholds for the two statistics to flag problematic arrays . in our practice
we set the cut - offs at 0.8 , 3 , and 6% for pearson correlation coefficient , z - value , and the outlier percentage , respectively .
then , common arrays on the two lists reported by mof satisfying the thresholds respectively can be considered primary candidates of problematic arrays . another way to set
the threshold is to take the top 15%20% worst arrays as indexed in the two lists , and then identify common arrays .
we would like to caution that the user needs to adjust the threshold based on their experience and the available resources . for reference , we report here two scenarios in using mof , illustrated with experimental data generated by our core facility . in one scenario ,
a couple of problematic arrays are so different from the rest of the arrays that the two statistics effectively singled them out .
for example , we have a dataset composed of 35 arrays using the universal human reference ( uhr ) rna sample ( stratagene , la jolla , usa ) , both indices flagged the same 3 arrays as outlier arrays ( figure 1 ) .
each of the 3 arrays had a distinctly lower average correlation coefficient and higher percentage of outlier spots than other arrays . in details ,
pearson correlation coefficients were 0.22 , 0.30 , and 0.49 for the top 3 arrays , and > 0.76 for the rest ; taking |z|=3 , the same 3 arrays contained outliers of 28%42% , with the fourth showing 11% and the rest showing 5% or less .
two of these three arrays were confirmed by the laboratory as failed experiments with reasons identified . in another scenario ,
there is a lack of outstanding problematic arrays and on the two lists reported by mof the statistical index changes relatively smoothly , from low to high for pearson correlation coefficient or from high to low for z - score reported outlier percentages .
for example , in a dataset composed of 185 arrays of human blood samples ( 30 controls and 155 patients with breast disease ) , following the thresholds we have about 30 arrays from either reported list to work on .
12 arrays were common in the top 30 arrays on both lists and were later verified to have obvious problems by scatter plots .
mof has been successfully applied to our in - house microarray data as a routine qa measure ( 6 ) .
however , users should note that in order to apply mof to any microarray datasets , the two assumptions mentioned above must be met . in our practice , when applying mof to a mouse dataset from different tissue types , tissue specific clusters were generated apparently due to the differences in gene expression profiles between tissue types .
in such a case , the first of the two assumptions was not satisfied and thus mof was not applicable .
as an additional note , the percentage of outliers changes when using different z - scores . selecting different z - scores may make some arrays to stand out with apparently higher percentages of outlier data points .
therefore , the relative level of percentage of outliers is a more meaningful characteristic for judgment than the absolute value itself .
users are encouraged to identify more realistic cut - offs in their specific settings using our thresholds as guidance .
although a minimal of three arrays is required for correlation coefficient to pick out an outlier array , we recommend that mof should be used on datasets comprised of ten or more arrays .
historical data obtained with the same technology on the same tissue type can be used in the analysis to increase the normal array base .
 we developed an r function named  
 microarray outlier filter  ( mof ) to assist in the identification of failed arrays . in sorting a group of similar arrays by the likelihood of failure , 
 two statistical indices were employed : the correlation coefficient and the percentage of outlier spots . 
 mof can be used to monitor the quality of microarray data for both trouble shooting , and to eliminate bad datasets from downstream analysis . 
 the function is freely avaliable at http://www.wriwindber.org / applications / mof/. 