magnetization transfer ( mt ) imaging is a quantitative approach for detecting subtle or occult abnormalities in brain tissue . in previous studies , the magnetization transfer ratio ( mtr ) , an index of mt imaging , was sensitive to brain changes in patients with mild cognitive impairment , an alzheimer 's disease prodrome [ 1 , 2 ] , to new lesions in patients with multiple sclerosis ,   and to changes associated with progression in chronic neurological disorders .
the higher magnetic field strength afforded by 3 t allows mt image resolution to be augmented compared with conventional mt acquisition at 1.5 t [ 57 ] .
we developed a high resolution mt technique to detect subtle changes in anatomically small , functionally eloquent brain structures .
the increased field strength affords whole - brain coverage with considerably thinner slices , potentially reducing partial volume artifacts .
however , even among healthy subjects , numerous factors may introduce variability in measures derived from magnetic resonance ( mr ) data , such as static field b0 signal dropout and rf nonuniformity .
measurement variation may be introduced by scan repetitions , repositioning at different time points , and image post - processing . moreover , 3 t may be susceptible to variation associated with increased field strength .
such variability may pose limitations when conducting clinical comparisons to differentiate normal and diseased brains or in developing statistically predictive algorithms .   to validate high resolution mt for detecting early disease or for monitoring progression in chronic neurological disease ,
it is necessary to collect information on normative values and to evaluate the reliability and reproducibility of the measurements when measured across time in healthy controls .
this investigation evaluated observer - agreement of high - resolution mt measurements determined from repeated brain scans of 9 healthy volunteers .
we evaluated the reliability and reproducibility of the high resolution mt measurements in 12 brain regions of interest ( rois ) , applied statistical measures to the data and used complex multivariate mixed - effects models to test the statistical significance of several effects due to region , subject , observer , time , and manual repetition .
the study was approved by the irb at the north shore university health system , and conducted following the ethical principles outlined in the declaration of helsinki .
eleven healthy adult volunteers were randomly selected from a database maintained at the center for advanced imaging , radiology department , northshore university health system provided written informed consent and evaluated for eligibility criteria . to protect the subjects ' confidentiality ,
all data were de - identified and handled according to the guidelines specified by the health insurance portability and accountability act ( hipaa ) in the usa .
brain images were acquired using a 3 t general electric ( ge ) hdx system ( waukesha , wi , usa ) .
each volunteer was scanned twice in a randomly - selected time interval between 1 to 4 weeks .
methods for reducing random errors in image acquisition included the use of a body - coil for excitation to control b1 non - uniformities and an 8-channel quadrature receive - only coil .
mt pulses with ( ms ) and without saturation ( m0 ) were applied at an offset frequency from water resonance . to accelerate the scan for whole - brain coverage ,
while maintaining thin slices , the image protocol was optimized based on 3 t using 3d spgr .
the gaussian sinc mt pulse was applied in 8  ms at a 1200  hz offset .
the stability of the scanner and set - up procedure were addressed with a fixed set of parameters per subject .
mt pulse was based on a three - dimensional spoiled gradient recalled ( 3d spgr ) acquisition .
the image protocol included the following parameters : tr 34 to 35  ms , te 4 to 8  ms , imaging fa 5 , bandwidth 15.6  khz , 0.75  nex , phase fov 0.75 , voxel dimensions 0.9  0.9  0.9 ~ 1.3  mm .
the whole brain was covered in 90 to 140 slices with acquisition time ranging from 7 minutes 40 seconds to 10 minutes 20 seconds using a partial k - space acquisition .
mtr maps were generated off - line on a general electric aw workstation ( general electric , milwaukee , wi , usa ) using the standard equation :  
 ( 1)mtr = m0msm0100% , 
							 where ms and m0 were the signal intensities in a given voxel obtained , with and without the mt saturation pulse , respectively .
the 12  rois were : genu , splenium , left and right hemispheres of the hippocampus , caudate , putamen , thalamus , and cerebral white matter .
each roi was sized approximately 30 to 43  mm and manually and independently placed by observers 1 and 2 ( authors s.s . and y.w .
after an initial consensus decision was drawn regarding the sizes and locations of the 12  rois , the observers performed manual segmentations of the roi independently on each set of images .
mtr values were extracted using the manually - defined rois with the combinations of observer , time point , and repetition ( table 1 ) . the mean and sds of the roi values were calculated .
meta - data were stored in a sas 9.1 ( sas , cary , nc , usa ) dataset , with individual volunteer identification numbers withheld and replaced by a sequence of 1 to 9 for each subject .  
statistical analyses were performed using sas 9.1 ( sas institute , cary , nc , usa ; http://www.sas.com ) .
proc univariate ,   proc means ,   proc corr ,  and  proc mixed .
let y = yijklm having the indices described in table 1 be a random variable representing the mean roi value .
for the mth roi , we first computed the sample mean and standard deviation of all mean roi values : 
 ( 2)mean^(ym)=ym=1nml=12k=12j=12i=19yijklm , sd^(ym)={var  ^(ym)}1/2={1nm1l=12k=12j=12i=19(yijklmym)2}1/2 , 
									 where nm = i 
j  k  l = 9  2 = 72 measurements and the operator  
the 95-percentile normality range was approximately within the following interval , with the following lower and upper bounds :  
 ( 3)(mean^(ym)2sd^(ym ) ,  mean^(ym)+2sd^(ym ) ) . 
									
the term  normality range  as used in europe , could be arbitrarily - defined according to the number of standard deviations away from the mean .
thus , it should not be viewed as the range of the entire dataset , but rather an interval useful for estimating the population value by one or several standard deviations away from the mean . here
the critical value of 2 was chosen as recommended by bland and altman .   additionally , we justified using a student 's t - distribution with nm  1 = 71 degrees of freedom . for any tail probability of /2 ( e.g. , 0.025 for a 95-percent normality range )
, we used the quantile of the corresponding to particular t - distribution , such that  
 ( 4)tnm11(12)=t711(0.975)=1.994 , 
									 this value happened to be close to the recommended multiplier of 2 .
we first explored and measured the concordance between the various measurements fully nonparametrically via spearman 's rank correlation coefficient .
suppose that we correlated the roi values by observers j = 1 and j = 2 , then denoted the marginal ranks , rijklm = ranki(yijklm ) and rijklm = ranki(yijklm ) , respectively , for all j  j with j = 1 and j = 2 .
the sample version of pearson 's product - moment correlation coefficient between the ranks of the data was equivalent to spearman 's rank correlation coefficient : 
 ( 5)cor^(rijklm , rijklm)=(nm/2)l=12k=12i=19(ri1klmri2klm)l=12k=12i=19ri1klml=12k=12i=19ri2klm{(nm/2)l=12k=12i=19ri1klm2}1/2{(nm/2)l=12k=12i=19ri2klm2}1/2,=l=12k=12i=19(ri1klmri2klm)(nm/2)ri1klmri2klm(nm/21)sd(ri1klm)sd(ri2klm ) . 
						 where  denotes ( l = 1k = 1i = 1ri1klm ) and  denotes ( l = 1k = 1i = 1ri2klm ) .   assuming that there was no presence of any ties since the roi values were of continuous random variables , the spearman 's rank correlation coefficient between observers j and j was 
										 ( 6)corr(rijklm , rijklm)=16l=12k=12i=19diklm2(nm/2)(nm2/41 ) , 
									 where the difference of an arbitrary pair of marginal ranks for observer j and j was denoted by diklm = rijklm  rijklm , for all j  j. consequently , all of the raw mean roi values were converted to their marginal ranks and the differences between the ranks of each observation on the two variables
spearman 's rank correlation coefficient was also computed for the roi values between any two different time points k = 1 and k = 2 .  
bar diagrams were made to display the spearman 's rank correlation coefficients between observers or time points for each roi .
we used the normalized measure of dispersion of a distribution to evaluate the reproducibility of the measurement .
the measure was the coefficient of variation ( cv ) , defined as the ratio of the sd to the mean . 

( 7)cv^(ym)=sd^(ym)mean^(ym ) , 
									 where both the numerator ( i.e. , sample sd ) and the denominators ( i.e. , sample mean ) in the above expression for cv are provided in ( 2 ) .
skewed data , such as those generated by an exponential distribution for which the underlying population mean and standard deviation would be equal , and thus the cv became 1 .
< 1 would generally represent low variability , and cv > 1 would represent high variability . as in ( 4 ) and ( 6 ) , further stratified computations of cv for different observers , time point , or repetitions were achieved using formulae similar to ( 7 ) . as overall variability was likely a result of the effects illustrated in table 1 .
we employed a multivariate mixed - effects regression analysis to direct model the roi values .  
a variance - component approach has advantages over many stratified analyses , especially studying studies with a limited sample size . here , because of the novel imaging modality using mt and 3 t acquisitions with labor - intensive manual segmentation procedures , large number of subjects would not have been feasible . to conduct an analysis of variance ( anova ) based on the various effects , a distributional assumption of normality was necessary and convenient
. we would demonstrate ( see section 3.4 ) that the normality assumption was generally satisfactory .   thus , we could then consider adopting a linear random - effects model with all pair - wise interactions , in addition to a third - order interaction term : 
										 ( 8)yijklm=m+si+oj+tk+rl  + sioj+sitk+sirj+ojtk  + ojrl+tkrj+ojtkrl+ijklm ,  i=1,  ,9 ,  j=1,2 ,  k=1,2 ,  l=1,2 . 
									
the effects represented the following : m as intercept , si as subjects , oi as observers , ti as time points , ri as repetitions , and ijklm as the error team .
a random - effects model assumed that each of the effects would have independent normal distributions with mean and variance .   if normality had failed and because the data were mean roi values that were positively - valued , we would recommend a box - cox transformation , h(yijklm ,  ) , of the outcome variable with an optimal power coefficient  [ 1719 ] .
note that the log - normal becomes a special case when the power coefficient  = 0 .
this normality transformation is given by : 
										 ( 9)yijklm=h(yijklm,)={yijklm1,0log  ( yijklm),=0i=1,  ,9 ,  j=1,2 ,  k=1,2 ,  l=1,2 .
a profile log - likelihood , llik of  given the observations yijklm , would be maximized to estimate an optimal box - cox transformation via a nonlinear minimization routine , where the log - likelihood was 
										 ( 10)llik(yijklm)=nmlog  { sd(yijklm)}+(1){i=1nmlog  ( yijklm)}+c , 
									 where c was a constant free of the power coefficient to be optimized .  
due to the limited number of subjects , however , even with an optimal normality transformation , over - fitting and non - convergence might be issues . alternatively
, we could regard all of the observers , time points , and repetitions as fixed and specify a mixed - effects model .
the significances of the sources of variability were tested via a restricted maximum likelihood ( reml ) approach . for our multivariate analysis ,
the significance threshold for two - tailed p - values was set if p  .05 .
stratified by the time points within each roi , a two - way anova was performed by regarding all of the observers , time points , and repetitions as fixed .
we specified a mixed - effects model for simplicity . due to the complexity of the variance components
for example , all subjects were segmented by the same observers who were from an entire population of observers . in other words ,
the subject effect was always assumed to be random , while the remaining effect ( e.g. , here the observer ) was assumed to be fixed .
we simplified our notations by only keeping the indices for the subject and observer effects of interest .
we decomposed the data as follows : 
 ( 11)yij=+si+oj+sioj+ij ,  i=1,  ,9 ,  j=1,2 , 
									 where the subject effect si was assumed to be random in an upper - case letter , which had a normal distribution with mean 0 and variance s , for all i = 1 , 
i = 9 ) ; the observer effect oj was considered to be a fixed effect in a lower - case letter , with the constraint j = 1oj = 0 , with the corresponding parameter to the variance being o = ( 1/(j  1))j = 1oj , for all j = 1 ,  , j ( here j = 2 ) ; the interaction term between the subject and the observer si  oj was the degree to which the jth observer departed from his or her usual rating tendencies for the ith subject , which had a normal distribution with a mean of 0 and variance so ;   the errors terms ij were assumed to have an independent and identical distribution ( iid ) normal distribution with a mean of 0 and variance e .
for the same ith subject , the effects are further assumed to be subjected to the constraint j = 1(so)ij = 0 over all of the observers .
shrout and fleiss gave the true definition of icc using the variance ratio of the subject variance over the total variance , with its estimated version using the quantities via anova ( table 3 ) : ( 12)icc=s2so2/(j1)s2+so2+e2,icc^(3,1)=bsmsemsbsms+(j1)ems .
similar to the analysis described above , we adopted a hybrid approach by considering two effects at once , with the subject effect always assumed to be random and the time point assumed to be fixed .
the associate model was given by 
										 ( 13)yij=+si+tk+sitk+ik ,  i=1,  ,9 ;  k=1,2 .
as in ( 12 ) , the estimated intraobserver agreement and its estimate were provided by : 
										 ( 14)icc=s2st2/(k1)s2+st2+e2,icc^(3,1)=bsmsemsbsms+(k1)ems , 
									 where the interaction term the interaction term between the subject and the time si  tk had a normal distribution with a mean of 0 and variance st .
we performed a sensitivity analysis by computing 6 different icc values shrout and fleiss previously proposed assumptions for iccs ( table 4 ) .
a sas macro , written by professor robert hamer , university of north carolina school of medicine , chapel hill , nc , usa ( http://www.bios.unc.edu/~hamer ) , was run to perform the various icc computations .
eleven healthy adults provided written informed consent to be evaluated and 9 underwent brain scans .
mean age of participants who received scans was 37.9  14.2 years ; 7 participants were men and 2 were women .  
the mean roi values varied across different region ( table 5 ) . the left and right hemispheres tended to yield similar results when the average over these healthy subjects was considered .
spearman 's rank correlation coefficients showed that a majority of correlations within each observer was above 0.5 , suggesting a moderate to high concordance ( figure 3 ) .
time point 2 tended to yield higher concordance between the observers , which suggested a possible learning effect over time ( figure 4 ) . due to
limited sample sizes in this pilot study , in figures 3 and 4 , we demonstrated the effect of observers by averaging over repetitions by each observer .
similarly , we demonstrated the effect of time points by averaging over repetitions at each time point .
overall , cvs ranged from 1.2% in the genu for observer 2 to 7.0% in the right hippocampus for observer 1 ( table 6 ) . since all of the cvs were within 7% , that is ,
the tests of the normal distribution assumption marginally using the shapiro - wilk test indicated that only occasionally ( e.g. , for left caudate , left and right putamen , and right hippocampus ) , this assumption was not met ( see table 7 ) .
therefore , it was reasonable to specify linear mixed - effects modeling and two - way anova reported in sections 3.5 and 3.6 .   at time
point 1 , iccs were greater than 0.7 in regions of genu , left and right putamen , whereas iccs were from 0.5 to 0.7 in regions of splenium , left and right hippocampus , left caudate , and right cerebral white matter ( table 8) .
these results indicated moderate to strong interobserver reliability . in comparison , at time point 2 , iccs were greater than 0.7 in regions of genu , splenium , left and right caudate , putamen and cerebral white matter , and left hippocampus and thalamus , while iccs were from 0.5 to 0.7 in right hippocampus and thalamus .
however , for some rois such as the left cerebral white matter , right caudate , right thalamus , iccs increased from 0.2 ( at time point 1 ) to 0.9 ( at time point 2 ) , making it difficult to determine whether this represents a learning effect .   at each time point , intraobserver agreement was at least 0.5 for a majority of the regions ( table 9 ) .    six different methods for generating iccs exhibited similar patterns for high vs. low reliability results in different rois ( table 10 ) .
these mathematical and statistical methods may easily be generalized to practical studies with larger sample sizes or to studies of patients with active disease .  
we acquired repeat brain measurements based on a high resolution mt imaging protocol at 3 t in 9 healthy adults .
our results indicate moderate to high reproducibility , supporting the validity of this method for further studies .  
overall , higher intraobserver reliability was observed at the second time point than that at the initial time point , suggesting a possible learning curve effect for both observers .
interobserver reliability was generally lower than intraobserver variability , suggesting a strong observer effect in this comparison , which may be a factor in future investigations using mt imaging .  
our analyses examined different aspects in a typical observer - agreement study , using measures for concordance , reproducibility , reliability , variance - component analysis , and multivariate analysis . in other studies , all or some of such methods may be considered . however , with a simpler study of either several observers , or one observer with several repetitions at different sessions or time points , then these scenarios may only require several of our methods .
only a small sample of healthy volunteers was evaluated in this initial pilot study .  
therefore , the generalization of the 95-percentile normality range may be limited with respect to the wider spectrum of brain mechanisms represented in the broader population .
for instance , demonstrating summary measures using all possible observer and time point combinations may not lead to meaningful interpretations in all cases .
nevertheless , since the technology is new , this research may provide useful pilot information for future investigations .
moreover , the statistical methods employed and illustrated here may easily be generalized to studies with larger sample sizes and diseased subjects .  
another limitation was that this study aimed to evaluate only the reproducibility and reliability , rather than the accuracy in a more comprehensive validation study . in the absence of a true gold standard , such as one based on digital phantoms where realistic variability may still not be simulated , or on histopathology , improved reliability
further research would benefit from a useful algorithm to perhaps statistically and optimally estimate the underlying spatial  ground truth  [ 22 , 23 ] .  
finally , future research may be directed to evaluating the diagnostic utility of high resolution mt for early detection of alzheimer 's disease , multiple sclerosis or other neurological disorders and for monitoring progression across the clinical course .  
 magnetization transfer imaging ( mt ) may have considerable promise for early detection and monitoring of subtle brain changes before they are apparent on conventional magnetic resonance images . at 3 tesla ( t ) 
 , mt affords higher resolution and increased tissue contrast associated with macromolecules . 
 the reliability and reproducibility of a new high - resolution mt strategy were assessed in brain images acquired from 9 healthy subjects . 
 repeated measures were taken for 12 brain regions of interest ( rois ) : genu , splenium , and the left and right hemispheres of the hippocampus , caudate , putamen , thalamus , and cerebral white matter . 
 spearman 's correlation coefficient , coefficient of variation , and intraclass correlation coefficient ( icc ) were computed . 
 multivariate mixed - effects regression models were used to fit the mean roi values and to test the significance of the effects due to region , subject , observer , time , and manual repetition . a sensitivity analysis of various model specifications and 
 the corresponding iccs was conducted . 
 our statistical methods may be generalized to many similar evaluative studies of the reliability and reproducibility of various imaging modalities . 