recent technological advances allow the rapid generation of vast quantities of molecular biological data . at the same time , the sequencing of the human genome and subsequent efforts to catalogue the variation within it have created opportunities for testing thousands of sequence variations for association with disease , behavioural traits and physiological markers .
such applications are appealing because of the relative lack of success , to date , of positional cloning strategies that start with family - based linkage mapping , most likely due to insufficient sample sizes to detect genes of modest effect .
the whole - genome association scan is an increasingly feasible study design in which the genotyped markers are sufficiently closely spaced to detect linkage disequilibrium ( ld ) with all aetiological variants , and well - powered sample sizes are more attainable .
some initial studies have been performed in special populations and in small samples of outbred populations ; genome - wide admixture scans are imminent and , ultimately , routine scans will be performed for common diseases in large cohorts of outbred populations .
array experiments measuring large numbers of transcription or expression levels are another form of genome - wide analysis that have become widespread .
although the effect sizes expected in these studies are large by comparison with disease association studies , the sample sizes are constrained by cost to be relatively small , so that both types of study encounter problems of statistical power ( table 1 ) .
expression levels can be regarded as quantitative traits under genetic control , so that both kinds of large - scale exploration can occur in genome scans for loci influencing expression levels , or phenome scans demarking the influence of genetic pathways .
primarily , there is the multiple testing problem , whereby the chance of an exceptional result increases with the number of tests performed , even when there is no true association . to alleviate this problem ,
two broad strategies have emerged : first , to devise more sensitive tests , so that the penalty for multiple testing is less severe ; and , secondly , to propose different measures of experimental error for which the interpretation of multiple testing is less serious .
furthermore , genome - wide analysis creates problems of computational and cost efficiencies on account of the large volume of data to be generated and analysed . here , some recent work addressing these problems is reviewed .
for the study design , work is summarised that minimises the cost of a study , while maintaining its power . for the analysis ,
methods are reviewed for improving sensitivity in the presence of multiple gene effects , by combining evidence across tests , and some methods for reducing the computational burden of permutation tests are discussed .
this review is mainly concerned with a whole - genome association scan , using single nucleotide polymorphisms ( snps ) , for a dichotomous disease status .
it will be clear , however , that many of the methods apply in other situations , in particular to array expression studies .
although there are important differences between these two applications -- including the number of expected true associations , sample size and effect size ( table 1 ) -- their common exploratory character suggests that further advances may arise from crossapplication of ideas between these areas .
for this reason , some methods developed for expression studies are reviewed ; there is also a discussion on whether they may be suitable for genetic association scans .
the objects of inference used will be ' genes ' , with the understanding that , in this context , this can mean snps , whole genes , haplotype blocks , transcript levels or other features .
large samples of unrelated individuals have become the design of choice for genome - wide association scans , because earlier concerns about population stratification have been largely allayed by empirical methods .
estimates of the total sample size are in the order of thousands . because the majority of genes are not associated with disease , it is uneconomical to genotype the whole sample for all genes .
sequential study designs , in particular a two - stage block design , have been proposed for reducing the total cost of a genome - wide experiment , which remains the main limiting factor preventing large - scale application . in a two - stage design ,
all of the genes are typed in a subset of the sample , with only the genes showing a trend of association being taken forward for genotyping in the remainder .
this directs resources towards true associations at an earlier stage , so that the available sample size is larger for genes with true effects .
the design parameters for a two - stage study include the total cost , total sample size , size of the first and second sub - samples and rejection criterion at the end of the first stage .
some of these parameters are constrained in advance , with the others then chosen to optimise some objective .
one approach is to consider the genotyping cost as fixed and then find parameters that give the most power . a general rule of thumb , considering a number of disease models and correlation structures between markers , is to allocate 75 per cent of resources to the first stage and then carry the most promising 10 per cent of markers to the second . here , the sample size is a function of the genotype unit cost and the number of markers , within the overall cost constraint .
it is more likely that the sample size is fixed ( say , to provide sufficient power to detect a single association ) and the goal is to minimise costs while achieving power close to that of the one - stage design . in many situations ,
the cost can be halved while keeping power within 1 per cent of the one - stage design ; thus , the total sample size can be calculated to achieve a certain power ( say 81 per cent ) in the one - stage design and parameters then optimised for a two - stage design .
considering a range of genetic models , a general guideline is to set the sample size of the first stage to have 97 per cent power for individual tests and carry forward all markers with nominal p - values less than 0.15 .
the sample size for the first stage can not be calculated without knowledge of the true effects , however , so a more practical approach is to consider the ranks of test statistics of the true effects . here
, it is shown that similar information to the one - stage design is obtained by genotyping all markers on 50 per cent of the sample and then genotyping the 10 per cent most promising on the remainder , resulting in a decrease of about 45 per cent in the number of genotypes .
again , the total sample size can be calculated for a one - stage design ; this last guideline is currently the most practical available and applies over a wide range of genetic models and correlation structures between markers .
an application of this strategy has been reported in which the primary constraint is the quantity of dna available for study subjects .
about 44 per cent of the sample had sufficient dna to be typed for all markers , with the remaining 56 per cent used for the second stage .
an important feature of this study is that the test statistics are calculated over the full sample , with adjustment made for the interim test .
this is in contrast to the simpler approach used in the simulation studies , in which test statistics were calculated separately for the two stages and their p - values combined into an overall significance .
analysing both stages at once makes more efficient use of information and will be the more powerful method for computing significance in the whole sample .
these can result in substantial cost savings , on average , but have yet to become widely adopted , owing mainly to logistical difficulties .
for example , the stopping criteria must be applied to each gene separately , but genotypes are often obtained in bulk in array format , which makes it difficult to apply sequential designs efficiently across many genes .
the two - stage designs are a compromise solution using frequentist inference , which also avoid the uncertainty in actual sample size that occurs with sequential inference .
for example , different genotyping technologies may be used in the two stages , with different unit costs , perhaps using dna pooling .
many analysis methods are available for genetic data , but a first pass through a genome - wide scan may normally consist of single - locus tests for trend , perhaps additionally with two - locus interaction tests .
several methods are now available that exploit the important feature that the majority of tested genes are not associated , but there are a small number of true , but weak , associations to be found .
these methods are useful both for establishing statistical significance more strongly than single - locus tests , and for informally suggesting sets of genes for follow - up study . in the traditional hypothesis - testing framework , each gene is tested individually and then a stepwise adjustment procedure is applied both to control the family - wise type-1 error rate ( fwer ) and to declare individual genes associated .
this approach , related to the bonferroni correction , achieves strong control of the fwer , which is the probability of at least one false positive being within the desired rate when there are any number of true positives .
this is generally considered to be too conservative for genome - wide studies , however , because we can tolerate a small number of false positives if most true positives are detected .
more preferable is weak control of fwer , which ensures that the probability of at least one false positive is within the desired rate only when there are no true positives .
this is desirable , because we must defend against the possibility of there being no true associations in the sample , but it allows us to tolerate some false positives if some true positives are present .
a joint test of multiple genes can maintain weak control of fwer and should reveal greater evidence for association from a set of genes , although perhaps with less specificity for individual genes .
this argument motivates the partial sum statistics , which are formed by obtaining test statistics ( typically tests from a contingency table ) for each individual gene and then forming the sum of the k largest statistics , where k is a fixed number called the length .
the significance of the sum can be assessed by a permutation test and an overall significance estimated over a range of lengths .
a more flexible alternative to the sum statistic is the truncated product of p - values . here
, the product is formed of all the p - values lower than a preset threshold , or the k smallest p - values . when the individual tests have the same distribution , the rank truncated product has equivalent power to the sum statistic , but is more balanced when the tests have different distributions .
this will occur , for example , when conducting haplotype - based tests on regions of different sizes , leading to tests with different degrees of freedom .
analytic distributions are known for independent tests , which have been used in simulation studies to show improved power for combined evidence methods compared with traditional corrections .
the present authors prefer the truncated product to the sum statistic on account of its balanced combination of different test , and also prefer to truncate on rank rather than threshold because the number of true gene effects is fixed across studies , whereas their p - values are random . the length k should be close to the actual number of true associations , but this is generally unknown
. a range of lengths could be tested , with the most significant length used to select genes for follow - up analysis ; but there is no formal basis for this strategy , and simulation studies show that it is capable of grossly over - or under - estimating the number of true associations .
a judicious choice of a fixed length , say k < 20 for a genome - wide association scan , is generally advisable provided that the tests are reasonably independent . when there is strong dependency between tests , such as in single - marker analysis of a dense genome - wide scan , then the variable - length approach can be used to establish statistical significance , but not to estimate the number of follow - up genes .
informally , genes would be followed up in rank order of significance ; and if the prior power is high , this will tend to identify the true associations in fact , formal adjustments based on the closure principle are available for individual tests , which allow strong control of fwer , but the primary use of truncated products is to show that the strongest associations indeed arise from true effects . in working with the summary p - value rather than the complete data ,
some information is lost , and a single analysis of the data may be more efficient .
a natural approach is to estimate all gene effects together in regression model . on the genome - wide scale ,
a fixed - effects regression is impractical , requiring estimation of many more parameters than there are observations .
therefore , several methods proposed for microarrays regard a gene as having a random effect , and model the distribution of gene effects by parametric forms that can be estimated .
a simple model is to assume a normally distributed effect around zero , although this may lack power when most genes have no effect .
the model can be extended by assuming that the effect variability comes from small and stronger effects , with inference based only on the stronger effects .
another alternative is a mixture of a zero - centred normal and a point mass at zero or , more generally , a mixture of three normals with respectively positive , zero and negative means . here , the zero - centred distribution is regarded as the null distribution , which allows for small nonzero effects to be regarded as uninteresting if there is sufficient evidence for stronger effects .
these approaches reduce the dimensionality of the inference while modelling the complete data , rather than summarising each gene before combining evidence .
these methods offer promise for genome - wide association scans , an important open question being the precision in estimating the random effects distribution when the number and size of true associations are small .
for example , a method for testing whether the overall distribution of p - values is uniform has very little power compared with the bonferroni correction when the number of true effects is small ( authors ' unpublished data ) .
another important issue is the choice of random effects distribution : current methods assume hierarchical or mixture normal distributions , but experimental geneticists have favoured gamma distributions .
a useful feature of the mixture distribution models is that they generate maximumlikelihood probabilities of membership to each of the mixture components , for each gene , which can be interpreted informally as posterior probabilities of association allowing individual genes to be selected for follow - up study .
when the assumptions underlying analytical distributions are not met , permutation tests are a popular method for computing significance levels . in a genome - wide association study ,
the problem is that genotypes are correlated due to ld ; indeed , the correlations are necessary for the design to be successful . the standard procedure is to reassign trait values among study subjects , while keeping their genotypes fixed , thereby preserving the correlation structure across the multiple genes and realising the exchangeability conditions for a valid test .
when performing thousands of tests on thousands of subjects , however , a permutation procedure using thousands of replicates becomes extremely time - consuming , with possible running times of days or weeks .
the accuracy of the permutation test can be improved by noting that the minimum p - value , sum statistic and truncated product can all be regarded as the extreme value of a large number of observations .
therefore , they should follow the extreme value distribution and by fitting the parameters of the distribution to the values observed in permutation replicates , more accurate significance levels are obtained . equivalently , fewer replicates are needed to reach a given accuracy .
the efficiency gain depends upon a number of factors , including the true significance level and the number of tests , and it is difficult to compute standard errors for the empirical p - values .
nevertheless , this approach has the advantage of being generally applicable and , importantly , the fitted distribution can be re - used in subsequent tests of the same genes in the same population
. this will be useful for studies based on a standard genome - wide marker panel , leading to substantial time savings over the long term .
lin considered score statistics from regression models , showing that it is sufficient to multiply the score contributions of each subject by a normal random deviate to generate a realisation from the null distribution .
alternatively , seaman and mller - mysock suggest sampling directly from the multivariate distribution for all the genes . the distribution can be estimated by considering the score test from a regression model that includes all the genes as predictors
. this estimation may be difficult when the number of genes exceeds the number of subjects for which the procedure may need to be applied piecewise to subsets of genes .
the approach of lin also requires the sample size to exceed the number of genes , but preliminary results suggest that it would be more robust than that of seaman and mller - mysock when applied across the whole genome .
both of these approaches require the analysis to be expressed as a score statistic from a regression model , which can be done in most situations but may require additional work by the user .
currently , lin 's method seems better suited to genome - wide analysis , whereas that of seaman and mller - mysock is more applicable and efficient in smaller - scale candidate gene studies .
a further approach is to assume that the sampled markers are representative of an ' effective number ' of independent tests [ 47 - 50 ] . after estimating this number -- for example , from the singular - valued decomposition of the genotype correlation matrix --
there is no formal basis for this approach , however , and studies based on real data indicate that the results are not always accurate ; indeed , there may be no such effective number after all .
this approach is not recommended ; however , if it is used , all significant results should be confirmed by a permutation test .
another perspective on the multiple testing problem is that the family - wise error rate is not the most appropriate measure , and that other measures should be used that have better sensitivity and specificity in genome - wide studies .
although weak control of fwer for the overall significance has been advocated , some error control for the single tests is also desirable . here
, two prominent alternatives are discussed : false discovery rates ( fdrs ) and posterior error rates .
the original fdr by benjamini and hochberg is the expected proportion of false positives among all positive results , with the proportion defined as zero if there are no positives .
that is , if r is the number of positive results in a study and v is the number of these that are false -- that is , do not arise from true gene effects -- then : fdr  =  e(vr|r  >  0)prr>0 .
subsequently , storey and colleagues have argued that the choice of the appropriate rate depends on how many positive results there are , and , furthermore , that the rate is only meaningful when there is at least one positive .
this motivates the positive fdr ( pfdr ) , defined as the expected proportion of false positives among all positive results , conditional on at least one positive at a given significance level : rather than setting a fixed pfdr rate to control , storey and colleagues suggest giving a value to each test that indicates what pfdr would result from declaring that test significant .
the follow - up tests can then be chosen based on joint consideration of the number of tests selected and the pfdr associated with them .
formally , the q - value associated with an individual test is defined as the minimum pfdr achieved when declaring all tests significant at the level of the test 's p - value .
a q - value can be estimated for each test in a genomewide experiment and follow - up tests selected from those with the lowest q - values .
this last stage is somewhat informal and may be driven by logistic and financial constraints .
a difficulty with fdr methods is that they control an expected proportion , whereas an investigator will be more concerned with the actual proportion of false positives within a study . some insight is gained by considering the variation in within - study false discovery proportion or false discovery variance .
let i be an integer with p(i ) the i - th smallest p - value from a set of m tests .
if the i most significant tests are declared positive , then mp(i ) estimates the maximum number of false positives .
the associated variance is mp(i)(1 - p(i ) ) ( because the truth of a positive test is a binomial outcome ) and the coefficient of variation is 1-p(i)p(i ) for the within - study false discovery proportion .
this is greatest when p(i ) is small , so , for a fixed set of p - values , this coefficient of variation is greatest when the fewest tests are declared significant .
this will occur when a low error rate is set , or when there are few true associations , or when the power is low . in genome - wide association scans , the number of true associations is expected to be small by comparison with the number of tests , so that the false discovery variance is relatively high in relation to the target rate , and the fdr approach may not be reliable for controlling the error rate within studies . in gene expression experiments , however , the number of true associations is somewhat higher and fdr methods are more appropriate for those studies .
study the within - study proportion of false discoveries and give procedures that keep the number ( or proportion ) of false discoveries within an upper bound with given probability .
the attraction of this approach is that one can limit the number of false positives with reasonable confidence , with the main disadvantage being increased computation .
it is uncertain how the false discovery proportion behaves when it falls outside the upper bound and , although this approach is attractive , further operating characteristics may be needed before it becomes more widely used .
the most significant tests are most likely to be the true positives , but fdr and q - values ignore this in favour of averaging the error rate across all significant tests . efron and colleagues
propose the local fdr as the posterior probability that a null hypothesis is true , given an observed statistic .
the local fdr is calculated as 0f0t0f0t+(1-0)f1 t , where 0 is the prior probability that the null hypothesis is true , t is a test statistic and f0 and f1 are the probability densities of t under the null and alternative hypotheses , respectively . 0 and
note , however , that when the true value of 0 is near one , as is likely in disease association scans , empirical estimates of 0 may be greater than one , which leads to a downward bias if these estimates are truncated at one .
thus , it is better to fix a prior estimate of 0 from genomic considerations such as the number of expected disease genes ( o(10 ) ) and the number of genes in the genome ( o(10 ) ) .
the q - value should be preferred if all positive tests will be followed up with roughly equal priority , which may be the case for a moderately powered study in which true and false positives are not well separated .
the local fdr is preferable if decisions to follow up positive tests are taken on a case by case basis , because it is a property of single tests rather than the whole set of positive tests .
this applies if there are a few very strong associations , together with some moderate ones , or if additional sources of evidence , such as biological plausibility , are taken into account , together with the statistical association .
this is the posterior probability that a null hypothesis is true , given a statistic at least as extreme as that observed .
it is calculated as where now f0 and f1 are the cumulative distributions . for known 0 and f1 and large number of multiple tests ,
the fprp is the same as the q - value , the main difference being one of context .
fprp is intended to be applied across multiple studies and calculated from prior models , whereas q - values are motivated by the within - study fdr and are usually estimated from data .
fprp is also mathematically complementary to the positive predictive value of a discriminant , again differing in context . because fprp is a property of a range of test statistics
, it is appropriate for setting guidelines for the reporting of significant results , based on assumed models for 0 and f1 .
this means that results can continue to be reported according to their p - values , but with modified thresholds of significance .
a known proportion of reported results will then be false ; however , for assessment of specific tests for follow - up , the local fdr is more relevant to investigators .
posterior error rates such as local fdr and fprp are gaining support because informed proposals can now be made for the prior probability of the null being true , based on genomic considerations . which of the various measures to use depends on the context .
error control ' indicates whether a method provides some measure of error : ( 1 ) type - i error ; ( 2 ) posterior probability of association ; ( 3 ) expected proportion of false discoveries in a series of tests . ' appropriate for ' indicates whether , in the view of the authors , a method is suitable for genome - wide association or expression studies , based on the factors in table 1 .
several aspects of the analysis of genome - wide studies have been discussed , including study design , analysis method and error control , all of which bear on the likelihood of successfully identifying gene effects .
there are some key aspects that have not been considered here , including selection and grouping of markers to be tested , population choice and data quality control . to some extent
, these issues are specific to the type of study ; this review has focused on the more general statistical issues that apply to most studies .
the field will continue to develop rapidly as more studies are completed and there is much scope for new methodology .
in particular , combinations of the current methods may prove to be fruitful -- for example , including combined evidence tests within a two - stage design .
there is no best method for all studies , because of their differing properties and aims , but this review has identified some of the questions that should guide the choice of analysis method . another important area for development , which has not been discussed here , will be the incorporation of evidence from several sources , including association studies , gene ontology annotation , information from model organisms and structural bioinformatics , to give a holistic appraisal of the effects of genetic variation .
is supported by the dutch diabetes research foundation , the netherlands organisation for health research and development and the juvenile diabetes research foundation international ( 2001.10.004 ) .
 recent developments in the statistical analysis of genome - wide studies are reviewed . 
 genome - wide analyses are becoming increasingly common in areas such as scans for disease - associated markers and gene expression profiling . 
 the data generated by these studies present new problems for statistical analysis , owing to the large number of hypothesis tests , comparatively small sample size and modest number of true gene effects . in this review , strategies are described for optimising the genotyping cost by discarding promising genes at an earlier stage , saving resources for the genes that show a trend of association . 
 in addition , there is a review of new methods of analysis that combine evidence across genes to increase sensitivity to multiple true associations in the presence of many non - associated genes . 
 some methods achieve this by including only the most significant results , whereas others model the overall distribution of results as a mixture of distributions from true and null effects . because genes are correlated even when having no effect , permutation testing is often necessary to estimate the overall significance , but this can be very time consuming . 
 efficiency can be improved by fitting a parametric distribution to permutation replicates , which can be re - used in subsequent analyses . 
 methods are also available to generate random draws from the permutation distribution . 
 the review also includes discussion of new error measures that give a more reasonable interpretation of genome - wide studies , together with improved sensitivity . 
 the false discovery rate allows a controlled proportion of positive results to be false , while detecting more true positives ; and the local false discovery rate and false - positive report probability give clarity on whether or not a statistically significant test represents a real discovery . 
Introduction
Study design
Analysis methods for multiple associations
Permutation testing
False discovery rates
Concluding remarks
Acknowledgements