confident peptide identification through tandem mass spectrometry ( ms ) is one of the most important components in ms - based proteomics .
for this reason , a great amount of effort has been invested to develop automated data analysis tools to identify peptides through tandem ms ( ms ) spectra . among available
because each search method uses a different algorithm and proceeds from a different view of what spectrum components contain the most critical information for identification , the search results for one spectrum from various search engines may differ significantly .
however , it is also well - recognized that such difference may be turned into positive use : it would be useful to find complementary engines and combine the results in an effective way to enhance peptide identification.(1 ) combining search results from different methods , if feasible , definitely bears the possibility to improve the peptide identification confidence via reducing noise and utilizing complementary strengths .
the difficulty in combining results from different search methods largely comes from the lack of a common statistical standard.(2 ) the importance of having a community standard has been stressed , and efforts in reaching such community standard have been invested . using iterative expectation - maximization ( em ) , keller et al .
( 3 ) proposed a statistical model to estimate the probability , determined through a global analysis of ms spectra from an experiment , for a given spectrum to have correct peptide identification . in principle , results from different search methods may go through the same analysis and thus compared . however , if after the statistical analysis two different methods report different confident identifications for the same spectrum , one ends up needing to invent an ad hoc rule to decide which identification should be kept .
furthermore , to use the em approach , one needs to assume or guess , without theoretical / statistical foundation , the forms of the score distributions for true positives and false positives .
this , unfortunately , must weaken the validity of any statistical significance assignment obtained from such type of analyses . in our recent work,(4 ) it was shown possible to calibrate the statistics ( e - value ) of various search methods to reach a universal standard that is in agreement with the fundamental definition of e - value . for a given query spectrum and quality score cutoff s , e - value
is defined as the expected number of hits , in a random database , with quality score being the same as or larger than the cutoff .
a realistic e - value assignment thus provides the user with the number of false positives to anticipate when setting a quality score threshold .
most importantly , this peptide - centric statistical calibration allows one to combine search results even if the top hits from various methods disagree . another possible approach
to establish common statistical standard is through equating the false discovery rate ( fdr ) ( 5 ) of various methods considered .
this approach , however , does not provide statistical significance for each peptide hit and thus is not directly applicable to peptide - centric combination of different search results . to be explicit
, one may refer from fdr the e - value of a peptide hit with score identical to the first false positive , but any peptide hits with score better than the first false positive can not have their e - value assigned .
furthermore , for peptides with scores falling in the range [ sk+1 ,  sk ] , where sk represents the kth best score of false positives , one can not distinguish them statistically well except by using some ad hoc interpolations .
there also exist other issues concerning misleading inference using fdr , but this is not the focus of the current paper and we refer the readers to a few relevant literatures . in this paper , in addition to providing a universal protocol to combine search results , we also carry out the performance assessment for all possible combinations , among seven database search methods , of two and three search methods using the receiver operating characteristic ( roc ) curves .
the database search methods employed in our analysis are sequest(8 ) ( v27 rev12 ) , probid(9 ) ( v1.0 ) , inspect(10 ) ( v20060505 ) , mascot(11 ) ( v2.1 ) , x !
tandem(12 ) ( v2007.07.01.2 ) , omssa(13 ) ( v2.0 ) and raid_dbs.(14 ) to better illustrate the main points of this paper , we have relegated to the supporting information a large number of roc curves that convey similar information of that exhibited in the plots of the main text .
since the centroid mode seems to be the dominant mode in ms database searches today , we present in the main text only results from centroid mode data and results from profile mode data are shown in the . throughout the paper ,
we use dalton ( da ) as the unit for molecular weight . in the following ,
we will then describe briefly in   the implementation , followed by our main results : best combinations within search methods tested .
in this section , we will start with the definitions of p - value and e - value , which will be frequently used for the rest of this paper .
we then describe the mathematical underpinnings of how to combine the p - values of different database search methods to result in a final e - value .
we should note that the mathematical formulation employed here was first introduced by fisher,(15 ) and its extensions and applications to other research areas also exist . to the proteomics community
let us define the p - value and e - value in the context of peptide identification in database searches .
for a given spectrum  and a score cutoff sc , one may ask what is the probability for a qualified ( with molecular weight in the allowed range ) random peptide to reach a score larger than or equal to sc .
this probability p(sc ) , a function of sc , is called the p - value . for spectrum
 , if a database contains n qualified , unrelated random peptides , one will expect to have e(sc ) = np(sc ) number of random peptides to have quality score larger than or equal to sc .
this expectation value e(sc ) is by definition the e - value associated with score cutoff sc .
if one further assumes that the occurrence of a high - scoring random hit is a rare event and thus can be modeled by a poisson process with expected number of occurrence e(sc ) , one may then define another p - value , which is called the database p - value , via 

 the database p - value pdb(sc ) represents the probability of seeing at least one hit in a given random database with quality score larger than or equal to sc .
note that , at the level of pdb , one may compare the statistics from different search methods using different sizes of random databases .
because of the differences in the choices of optimal search parameters , it is likely that different search methods , for the same query spectrum , may search over different number of qualified peptides , that is , having different effective database sizes
. therefore , combining the database p - values is the natural choice if one were to merge results from different search methods .
suppose that one wishes to combine the search results from l different search methods , each peptide candidate will have in principle l different p - values reported by the l search methods .
the formula in eq 6 of the next subsection provides us the final combined p - value pcomb from the list .
once pcomb is obtained , we may invert the formula in eq 1 to get a combined e - value ecomb via 

 having outlined how to obtain the final quantity of interest , ecomb , we now turn to the mathematical underpinnings of how to combine a list of , ideally independent , p - values reported by different database search methods .
consider an event labeled by a list of l independent quantities s1 ,   s2 , ... ,  sl . each quantity si may have an associated p - value pi depending on the statistics of the variable si .
an important issue to address is how one should combine all the pi values to obtain an overall p - value . in the context of combining search results of different methods to assign statistical significance to a certain candidate peptide 
pl ) uniformly distributed in the interval ( 0 ,  1 ) ] , what is the probability of finding their product to be smaller than a certain threshold . to put it in a more concrete framework , one may consider a unit hypercube whose interior points having coordinates ( x1 ,  x2 , ... ,  xl ) with 0  xi  1 for all 1 
i  l. one then asks what is the volume bounded by the hypersurfaces xi  0 and ( i=1lxi )   with  = i=1lpi .
we may express this volume f( ) mathematically as an integral : 

 where (x ) is a step function with (x > 0 ) = 1 and (x < 0 ) = 0 .
let f( )  f()/ , we have 

 with (x ) being the dirac delta function that takes zero value everywhere except when x = 0 where its value approaches infinity .
for the ease of computation , we make the following changes of variables :   e and xi  e. after this change , all the new variables t and ui are in the range ( 0, ) .
equation 4 now becomes ( with  = e understood ) 

 where the identity (e  e ) = e(t  c ) is used . using the integral expression of the delta function
, 

 we may rewrite eq 4 as ( with  = e understood ) 

 where the last equality results from choosing the integration path to enclose the lower half of the complex k plane .
we may now go back to f( ) by integrating f( ) . 

 with 
= i=1lpi while combining the lp - values p1 ,  p2 , ... ,  pl . as specific examples , when l = 2 , we have f(p1p2 ) = p1p2[1  ln ( p1p2 ) ] , and when l = 3 , we have f(p1p2p3 ) = p1p2p3[1  ln(p1p2p3 ) +  ln(p1p2p3 ) ]
. we will provide in the   more examples to elucidate the consequence of the formula provided in eq 6 .
 confident peptide identification is one of the most important components in mass - spectrometry - based proteomics . 
 we propose a method to properly combine the results from different database search methods to enhance the accuracy of peptide identifications . 
 the database search methods included in our analysis are sequest ( v27 rev12 ) , probid ( v1.0 ) , inspect ( v20060505 ) , mascot ( v2.1 ) , x ! tandem ( v2007.07.01.2 ) , omssa ( v2.0 ) and raid_dbs . using two data sets , one collected in profile mode and one collected in centroid mode , we tested the search performance of all 21 combinations of two search methods as well as all 35 possible combinations of three search methods . 
 the results obtained from our study suggest that properly combining search methods does improve retrieval accuracy . 
 in addition to performance results , we also describe the theoretical framework which in principle allows one to combine many independent scoring methods including de novo sequencing and spectral library searches . the correlations among different methods are also investigated in terms of common true positives , common false positives , and a global analysis . 
 we find that the average correlation strength , between any pairwise combination of the seven methods studied , is usually smaller than the associated standard error . 
 this indicates only weak correlation may be present among different methods and validates our approach in combining the search results . 
 the usefulness of our approach is further confirmed by showing that the average cumulative number of false positive peptides agrees reasonably well with the combined e - value . 
 the data related to this study are freely available upon request . 