functional magnetic resonance imaging ( fmri ) studies in chronic pain patients have the potential to provide valuable information about the neural mechanisms and efficacy of analgesic therapies , including drug treatments , acupuncture , brain stimulation , distraction tasks , mindfulness meditation , and cognitive - behavioral interventions .
furthermore , fmri studies in healthy individuals have provided insights into neural mechanisms of pain modulation , such as the placebo effect and conditioned pain modulation .
such studies rely on the assumption that brain responses to pain are consistent in sessions conducted on separate days before , during , and after therapeutic interventions . despite this common practice ,
only one study specifically addressing the intersession reliability of pain - related fmri activation has been published .
taylor and davis ( 2009 ) examined the spatial reliability of fmri activation associated with painful mechanical stimulation of the hand in 6 subjects across four biweekly sessions , finding high across - session spatial repeatability in second somatosensory cortex ( s2 ) , but lower and more variable spatial repeatability in primary somatosensory cortex ( s1 ) and thalamus ( taylor and davis , 2009 ) ; other areas that are part of the cortical network classically activated by pain ( as reviewed in duerden and albanese , 2013 ) were not examined in the study .
furthermore , studies examining across - session reliability of the amplitude ( percent signal change ) of pain - related fmri responses have not yet been published .
gaining a better understanding of the stability of repeated , intersession measures of responses to pain ( both spatial extent and bold signal amplitude ) in the entire cortical pain network will enhance the ability to interpret data collected in studies of pain - reducing manipulations .
thus , characterization of test  retest reliability of pain - related fmri activation is a critically important issue to address .
high test - retest reliability of fmri responses across two or more sessions has been reported for a wide variety of tasks , including motor , auditory detection , language , learning , and memory ( atri et al . , 2011 ; bennett and miller , 2010 ; cacares et al . , 2009
gountouna et al . , 2010 ; havel et al . , 2006 ; kiehl and liddle , 2003 ; maitra et al . , 2002 ; mcgregor et al . , 2012 ; yoo et al . ,
retest reliability of fmri responses has been found for other tasks such as mouth movements ( havel et al . , 2006 ) , reward ( fliessbach et al . , 2010 ) , and spatial attention ( gorgolewski et al . , 2013 ) . while these results may reflect differences in reliability across tasks
approaches have included measuring spatial extent or spatial overlap of significant task - related activation at the individual or group level , performing voxelwise group - level contrasts of the blood oxygenation level - dependent ( bold ) response , and calculating voxel - wise intraclass correlation coefficients ( icc ) based on the amplitude of the bold response .
the first objective of this study was to characterize the intersession reliability of pain - related fmri activation elicited by painful contact heat stimuli in healthy volunteers , considering both spatial extent and amplitude measures . to address the possibility that reliability of pain - related fmri responses may vary across brain regions ,
the study separately evaluated reliability of responses in cortical areas that are part of the network classically activated by pain ( as reviewed in duerden and albanese , 2013 ) , including : s1 , s2 , pregenual anterior cingulate cortex ( pacc ) , anterior midcingulate cortex ( amcc ) , insular cortex ( ins , distinguishing anterior and posterior ) , supplementary motor area ( sma ) , and several frontal lobe regions ; the thalamus was also examined .
stimulus paradigms using painful heat vary across studies , with some protocols using fixed temperatures for all subjects ( resulting in highly variable perceived pain intensity reports across the group ) and others using subject - specific temperatures that produce consistent perceived pain intensities across the group .
a recent study reported no difference in pain - related fmri activation produced by fixed temperature stimuli and individually - determined contact heat pain stimuli ( van den bosch et al . , 2013 )
; however , the question of whether these different stimulus paradigms produce equally reliable results across sessions remains unanswered .
a second objective of this study was to compare the reliability of pain - related fmri responses for fixed temperature stimuli with that of subject - specific temperatures to address whether one or the other approach provides for more reproducible results .
the subjects in this study also performed a simple motor task ( finger - thumb opposition ) in the same sessions as they experienced painful heat . to evaluate the appropriateness and validity of the analytical approach used to assess intersession reliability of pain - related fmri responses , this study used the same approach to assess
the across - session reliability of motor - related fmri activation and compared the results with those previously published .
fourteen subjects ( mean age 44.3  years , sd 19 , range 2275 ; 7 male ) participated in the study .
all subjects were healthy , with no major medical , neurological , or chronic pain disorders .
young female subjects were tested during the follicular phase of their menstrual cycle ( days 3 to 10 ) to reduce variance potentially related to effects of gonadal hormone fluctuations on pain perception . all postmenopausal women
the protocol for this study was approved by the university of maryland institutional review board for the protection of human subjects .
painful heat stimuli were delivered to the left dorsal forearm of each subject using an mr - compatible peltier thermal probe with a 2.6  cm contact surface ( tsa - ii , medoc ltd . , israel ) .
two temperatures of painful heat were delivered to each subject : ( 1 ) 48  c and ( 2 ) a subject - specific temperature perceived as moderately painful , which was defined as the temperature the subject rated as 50 on a 100-point computerized visual analog scale ( vas ) for pain intensity .
these stimuli will be referred to as 48  c and 50vas throughout this manuscript .
temperatures that evoked the perception of moderate pain ( 50vas ) ranged from 47.5 to 50.0  c ( mean 49.0  c , sd 1.0 ) .
subjects participated in a training session in a laboratory room dedicated to psychophysical assessments at least one day prior to scanning . during the training session
, subjects were first presented with an ascending series of thermal stimuli ranging from 42 to 50  c .
each temperature was presented for 15  s ( including ramp up and down time at a rate of 2.7  c / s ) , followed by a 30-s interstimulus period of nonpainful warmth ( 37  c ) .
subjects were then presented with a series of heat stimuli expected to be in the painful range ( 4650  c ) , with temperatures presented twice each in a randomized order . after each stimulus , subjects used an mr - compatible trackball ( fellowes , http://www.fellowes.com ) to rate peak pain intensity on a computerized vas , which consisted of a vertical scale labeled  no pain  at the bottom and  most intense pain imaginable  at the top ( dapsys , brian turnquist , johns hopkins university , http://www.dapsys.net ) .
individual subject ratings for the range of temperatures were used to interpolate the subject - specific temperature that evoked a perception of moderate pain ( 50vas ) and to confirm that the 48  c stimulus was perceived as painful .
each subject participated in three scanning sessions conducted on separate days , with the mean interval between sessions 15  days ( sd 18 ) .
the high variance in the between - session interval was mainly attributable to two women who were scanned across multiple months to ensure testing was conducted during the follicular phase of the menstrual cycle .
most scans occurred between 4 and 9 pm and lasted about 90  minutes ; each of the 3 sessions for an individual subject began at about the same time each day to reduce circadian variability in perception and hormone levels . during each scanning session ,
information about functional brain responses was collected using bold fmri and information about brain anatomy was collected using mr imaging ( details below ) .
the fmri portion of the session consisted of two scans in which painful heat stimuli were delivered , separated by a 30-minute interval .
the painful heat stimulus protocol for each fmri scan consisted of delivering the two temperatures ( 48  c and the subject - specific temperature perceived as moderately painful ) six times each in a randomized order .
each temperature was presented for 15  s ( including ramp up and down time at a rate of 2.7  c / s ) , followed by a 30-s interstimulus period of nonpainful warmth ( 37  c ) . after each stimulus ,
the computerized vas was presented to the subject through mr vision 2000 goggles ( resonance technologies , van nuys , ca ) and the subject rated peak pain intensity using the mr - compatible trackball .
the duration of each stimulus cycle was 45  s : painful heat application ( 15  s ) , vas rating task ( 15  s , or less if the subject responded more rapidly ) , and rest period ( 15  s , or more if the subject completed the rating task in less than 15  s ) .
in the 30-minute interval between pain fmri scans , subjects rested quietly for approximately 10  minutes , performed a simple motor task ( right hand finger - thumb opposition at approximately 1  hz ) in a block design ( 24  s opposition alternating with 24  s rest ) for 6  minutes and 54  s while fmri data were acquired , then rested quietly for the remainder of the interval .
images were collected using a 1.5 tesla phillips eclipse scanner ( phillips healthcare , cleveland , oh ) .
functional mr images were acquired using a single - shot echo planar imaging t2 * -weighted sequence with an echo time ( te ) of 35  ms and flip angle of 90. acquired image resolution was 3.2    3.2  mm over a 24-cm field - of - view ( fov ) .
the images were zero padded to 128    128 pixels to provide a resolution of 1.875    1.875  mm .
the repetition time ( tr ) of 3  s provided full brain coverage using 24 axial slices ( 6  mm thick with no gaps between slices ) prescribed parallel to the anterior - posterior commisural plane .
high - resolution t1-weighted volumetric scans ( 4.5  ms te , 29  ms tr , 110 slices , slice thickness 1.5  mm , 0.938    0.938  mm in - plane resolution , 24-cm fov ) were acquired in the same plane as the functional images for anatomical detail . image processing was performed using the software package analysis of functional neuroimages ( afni ; cox , 1996 ) .
the first four volumes of each functional scan were discarded to exclude images acquired prior to stabilization of the magnetic resonance signal .
data from the 2 functional pain scans conducted within each session were concatenated for analysis .
functional images were motion - corrected by spatially registering the volumes from all functional scans to the first remaining volume ( afni routine 3dvolreg ) . to minimize effects from possible spike - related artifacts , signals greater than 2.5 sds of the overall bold signal were reduced ( afni routine 3ddespike ) .
time series were temporally smoothed to reduce high frequency noise using a moving 3-point weighted ( 0.15 - 0.70 - 0.15 ) average .
images were spatially smoothed to increase the signal - to - noise ratio using a 5-mm full width half - maximum gaussian kernel .
trends in the time series ( linear , second - order , and third - order ) were removed on a voxelwise basis to reduce low frequency noise components .
functional and anatomical images were transformed to common space ( tailairach and tournoux , 1988 ) , and the voxels resampled to 2    2    2  mm .
voxelwise normalization was performed by dividing the signal intensity at each time point by the voxel s mean intensity .
for each individual subject , a general linear modeling ( glm ) approach was used to identify brain regions in which the time course of the bold signal was significantly related to the task , either the painful stimulus or the motor task ( afni script 3ddeconvolve ) .
the glm for the pain scans consisted of three temporally independent regressors ( one for each temperature of painful heat and for the vas rating task ) each represented by a delayed boxcar function convolved using the afni block function to account for hemodynamic delay .
though the glm included a regressor for the vas rating task , results for this regressor are not presented because brain activity during the vas rating task is not a variable of interest in this study .
the glm for the motor scans consisted of a regressor for the motor task as well as motion correction parameters .
voxelwise regression of the bold signal time course with the appropriate model resulted in statistical parametric maps for pain - related activation and motor - related activation for each individual subject .
group maps of significant pain - related activation were calculated separately for each stimulus type ( 48  c and 50vas ) using regression coefficients from the glm , collapsed across sessions , in a one - sample t - test ( afni routine 3dttest ) .
the resulting group statistical parametric maps ( one for the 48  c stimulus and one for the 50vas stimulus ) were thresholded to identify significant activation associated with each type of painful heat using a cluster threshold approach to correct for multiple comparisons across the brain .
the cluster threshold criterion was determined using monte carlo simulations to estimate the likelihood of detecting false positives over multiple comparisons ( afni routine 3dclustsim ) .
based on the simulations , which were derived from whole - brain analyses , a significant cluster ( overall corrected p 
<  0.05 ) was defined as a minimum cluster size of 190  mm of contiguous voxels , each with a voxelwise p  <  0.05 . the cluster threshold criterion was applied to maps for the t - statistic for each stimulus type ( 48  c and 50vas ) to identify the voxels that responded significantly to each type of painful heat , resulting in two thresholded maps for each subject : ( 1 ) voxels responding significantly to 48  c stimuli and ( 2 ) voxels responding significantly to the 50vas stimulus .
statistical parametric maps for the motor task were thresholded using the same approach to identify voxels responding significantly during the motor task .
the pain - related and motor - related maps that resulted from this stage of analysis differed in spatial extent .
accordingly , cluster size thresholding for subsequent analyses ( described below ) involved different spatial threshold criteria for the pain- and motor - related activation maps . to evaluate reliability of pain- and motor - related fmri activation across the three sessions conducted in this study ,
two measures were calculated for voxels that showed significant pain - related activation , separately for each stimulus type ( 48  c and 50vas ) , or significant motor - related activation : ( 1 ) spatial reliability coefficients based on spatial localization and extent of activation and ( 2 ) intraclass correlation coefficients ( icc ) based on the glm regression coefficients , which reflect fmri response amplitude .
spatial reliability coefficients are based on the number of voxels commonly activated in all sessions ( rombouts et al . , 1998 ) ; voxels significantly activated by painful stimuli or the motor task were identified in each session using regression coefficients from the glm in a one - sample t - test .
spatial reliability coefficients were calculated separately for the 48  c stimulus , the 50vas stimulus , and the motor task , using the formula r = ( 3  number of voxels commonly activated in all 3 sessions)/(sum of activated voxels in all 3 sessions ) ( havel et al . , 2006 ) .
spatial reliability coefficients for the painful stimuli were also calculated across the two stimulus types ( 48  c and 50vas ) using the formula r = ( 2  number of voxels commonly activated by both stimulus types)/(sum of activated voxels by both stimulus types ) .
spatial reliability coefficients can range in value from 0 to 1 , with 0 indicating poor spatial reliability and 1 indicating perfect spatial overlap across sessions or conditions .
iccs were calculated using the regression coefficients from the glm ( which reflect bold response amplitude ) separately for the 48  c stimulus , the 50vas stimulus , and the motor task , resulting in three reliability maps ( one for the 48  c stimulus , one for the 50vas stimulus , and one for the motor task ) .
iccs were calculated separately for each painful stimulus type and the motor task ( afni routines 3dcalc and 3dmean ) using the formula described in mcgraw and wong ( 1996 ) for the degree of absolute agreement among repeated measurements : icc(a,1)(bmsems)bms  +  ( k1)ems+  kn(wmsems ) the icc therefore measures the correlation of the magnitude of pain- or motor - related fmri responses between sessions using a two - way mixed anova framework , where the variance is divided into between subject variance ( bms ) , within subject variance ( wms ) , and residual sources of variance ( ems ) , k is the number of repeated sessions , and n is the number of subjects within a session . an f - statistic and p - value associated with each voxel s icc were calculated based on the approach described in ( mcgraw and wong , 1996 ) .
the f - statistic maps were then thresholded using a cluster threshold approach to correct for multiple comparisons across the voxels that survived the initial thresholding step ( described above ) ( afni routine 3dclustsim ) .
as previously noted , the pain- and motor - related maps differed in spatial extent after the initial thresholding step ; as a result , different cluster size threshold criteria were calculated for these maps in these subsequent analyses . for pain - related maps , a significant cluster ( overall corrected p  <  0.05 )
was defined as a minimum cluster size of 127  mm of contiguous voxels , each with a voxelwise p  <  0.05 . for motor - related maps , a significant cluster ( overall corrected p  < 
0.05 ) was defined as a minimum cluster size of 148  mm of contiguous voxels , each with a voxelwise p  <  0.05 . the thresholding step was conducted separately for each stimulus type and for the motor task , resulting in three statistical parametric maps ( one for the 48  c stimulus , one for the 50vas stimulus , and one for the motor task ) of brain areas with significantly reliable pain- or motor - related activation .
these maps were then masked to exclude voxels where within - subject variance contributed to more than 1% of the total variance ( calculated using afni s 3dicc_reml routine ) .
the purpose of this step was to eliminate voxels with artifactually high iccs ( such as would be the case if bms was high , wms was high , and ems was low ) , thereby reducing the possibility of false positives ( chen et al . , 2007 ) .
ems represents random variance with unknown sources that might include mri - related noise , physiological noise , or cognitive processes unrelated to the task that change over time ( bennett and miller , 2010 ) .
significantly reliable pain - related activation ( as determined by clusters with significant iccs ) associated with each stimulus type was examined in anatomically - defined regions of interest ( roi ) known to be involved in processing painful stimuli : the arm representation area of s1 , s2 , pacc , amcc ( corresponding to the area referred to as the mid - acc in many previous pain studies ) , anterior ins ( ains ) , posterior ins ( pins ) , sma , inferior frontal gyrus ( ifg ) , medial prefrontal cortex ( mpfc ) and dorsolateral prefrontal cortex ( dlpfc ) .
the arm area of s1 was defined as the region of the postcentral gyrus starting from the most medial portion of the hand representation ( delineated by the  knob  created by the postcentral gyrus ) and extending approximately 2  cm medially along the surface from that point , excluding digit representations ( servos et al . , 1998 ;
van westen et al . , 2004 ) ; these boundaries should encompass the complete arm representation .
the boundaries of the pacc and amcc were delineated based on vogt ( 2005 ) .
the boundaries of the other rois were described previously ( moulton et al . , 2005 ) .
significantly reliable motor - related activation ( as determined by clusters with significant iccs ) was examined in two anatomically - defined motor rois : the hand representation area in the primary motor cortex ( m1 ) and sma . for the painful stimuli ,
separate evaluations were conducted for portions of each roi contralateral ( right hemisphere ) and ipsilateral ( left hemisphere ) to the stimuli . for each stimulus type ,
the largest cluster of significantly reliable ( as defined by significant icc values ) pain - related activation was identified in each roi .
the reliability of each cluster was then classified using the peak icc value based on the conservative criteria described by shrout ( 1998 ) : virtually no reliability ( 00.1 ) , slight reliability ( 0.110.4 ) , fair reliability ( 0.410.6 ) , moderate reliability ( 0.610.8 ) , and substantial reliability ( 0.811 ) . the motor data were analyzed using the same approach , with the largest cluster of significantly reliable motor - related activation identified in each roi contralateral to the hand performing the task ( left hemisphere ) and ipsilateral ( right hemisphere ) . to address the question of whether a painful stimulus of constant temperature or of constant perceived intensity produced more reliable bold fmri responses , a group - level contrast between significantly reliable pain - related activation for the 2 stimulus types was conducted in which the voxelwise icc values for each stimulus type were contrasted ( donner and zou , 2002 ) . to address the question of whether the perceived pain intensity - dependent response is consistent across sessions
, a voxelwise analysis was conducted in which individual pain intensity ratings for each subject and each stimulus were used in the glm .
the analysis was conducted on the 48  c pain condition , which was the protocol in which ratings varied the most across subjects . for each individual subject and each session ,
pain intensity ratings of each 48  c stimulus were used in the regression model for each voxel , resulting in parameter estimates for each subject at each voxel that represented the degree to which perceived pain intensity covaried with the bold response .
the parameter estimates were then used in a group analysis to identify voxels where the pain intensity ratings significantly predicted the magnitude of the bold response to the stimulus .
intersession reliability in these voxels could then be evaluated by calculating iccs . to address the question of whether the duration between sessions had an effect on intersession reliability , we reanalyzed our pain and motor task data sets to calculate iccs on task - related activation detected when duration between session was used as a covariate in the analysis .
the first analysis stage involved conducting a voxelwise glm repeated measures ancova for each of our pain and motor tasks separately , using each subject s average days between sessions as a time - invariant covariate using the methodology of winer ( 1971 ) .
the analysis excluded voxels that violated the assumption of homogeneity of regression slopes because ancova is not an appropriate statistical test when this assumption is violated ( tabachnick and fidell , 2007 ) .
our homogeneity test revealed that 98% of voxels across the brain did not violate this assumption and were therefore included in the ancova . in the second stage of the analysis
, iccs were calculated based on the anova framework described by mcgraw and wong ( 1996 ) , but with a slight modification : a two - way mixed design of absolute agreement was used to calculate iccs on a voxelwise basis using adjusted variability estimates and degrees of freedom derived from the ancova analysis ( afni routines 3dcalc and 3dmean ) .
subsequent analysis steps involved the same thresholding , masking , and statistical steps as the original icc analysis .
pain intensity ratings obtained in the scanner were evaluated separately for each stimulus type using the nonparametric friedman test for intersession effects .
mauchly s test of sphericity was used to compare the error variance associated with each stimulus type .
pain intensity ratings did not differ significantly across sessions for either stimulus type ( p  >  0.05 , friedman test , fig .  1 ) , indicating that within the parameters of this study , repeated testing did not change perceived intensity of the painful heat stimuli .
variance of pain intensity ratings was greater for the 48  c stimulus than the perceptually - equalized 50vas stimulus ( p  <  0.05 ) .
significant pain - related activation was found in expected brain regions of interest ( roi ) for both types of painful heat stimuli ( fig .  2 ) .
most prescribed rois showed significant increases in the bold signal associated with painful heat , including pacc , amcc , ains , and s2 .
a few rois , including the arm region of s1 and frontal lobe regions , showed significant decreases in the bold signal in response to painful stimuli ( fig .  2 ) .
the map of pain - related activation from this study was compared with the reverse inference meta - analysis image from the neurosynth database ( yarkoni et al . , 2011 ) and was found to have notable overlap , particularly with respect to the insula , anterior cingulate cortex , and some dorsolateral frontal regions . as reported above , robust activation was found in several other brain regions that were not identified in the image from the neurosynth database , but have been described in the literature ( duerden and albanese , 2013 ) .
intersession reliability of the spatial extent of significant pain - related activation is shown in fig . 
spatial reliability coefficients were low ( <  0.2 ) in most pain - related rois , for both the 48  c stimulus ( table  1 ) and 50vas stimulus ( table  2 ) .
spatial reliability differed by roi , with the highest spatial overlap shown for both stimulus types in the ains and no voxels displaying complete overlap ( e.g. , significant activation in all 3 sessions ) for either stimulus type in the pacc .
spatial overlap between the two stimulus types ( 48  c and 50vas ) is shown in fig . 
spatial reliability coefficients were relatively high ( most >  0.5 , table  3 ) , indicating moderately consistent spatial extent of activation regardless of whether the stimulus was of constant temperature or of constant perceived pain intensity .
spatial overlap between the stimulus types showed regional differences , with the highest reliability coefficients in the ains and low reliability coefficients in the pacc .
reliability of pain - related activation amplitude was assessed in a two - step process ( see methods ) , by calculating voxelwise iccs and then applying a statistical filter to eliminate voxels with artifactually high icc values due to a combination of high wms and low ems .
5a shows the average bold response amplitude for each session in the voxel in the left ifg with the highest icc prior to the statistical filtering step .
the graph shows a highly variable pain - related bold response in this voxel across sessions ; however , the icc value ( 0.75 ) calculated for this voxel was the highest in the roi and statistically significant .
the high icc value obtained for this voxel is attributable to a combination of high bms , high wms and low ems .
this example illustrates that it is possible to obtain a high icc value for a voxel despite high variability ( low reliability ) of the response from session to session .
thus , relying entirely on icc values without examining the variances that contribute to the icc can result in a voxel being identified as having a reliable intersession response when in fact it does not ( in other words , a false positive ) .
the statistical filtering step employed here , which eliminates voxels with high wms ( such as the voxel shown in fig . 
5b shows the average bold response for each session in the voxel in the left ifg with the highest icc after the statistical filtering step .
this voxel , which is anatomically close to the peak voxel found prior to statistical filtering , shows much more consistent and reliable pain - related responses across session , and an icc value ( 0.73 ) that more accurately reflects the low intersession variance .
intersession reliability of pain - related activation associated with the 48  c stimulus is summarized in table  4 and fig . 
table  4 identifies the largest cluster of significantly reliable pain - related activation found in each roi and the icc value associated with the peak voxel in the cluster ; additional clusters were also found in most rois .
fair - to - moderate reliability ( based on shrout , 1998 ) was found in every roi except ( 1 ) contralateral pacc , which showed slight reliability , and ( 2 ) s1 , pins , and ipsilateral thalamus , which contained no clusters that were significantly reliable across the three sessions .
rois with the highest iccs ( the upper portion of the moderate range ) included amcc , ains , and several frontal lobe areas .
clusters of significantly reliable pain - related activation from selected rois are shown in fig . 
intersession reliability of pain - related activation associated with the 50vas stimulus is summarized in table  5 and fig . 
table  5 identifies the largest cluster of significantly reliable pain - related activation found in each roi and the icc value associated with the peak voxel in the cluster ; additional clusters were also found in most rois
. fair - to - moderate reliability ( based on shrout , 1998 ) was found in every roi except ( 1 ) ipsilateral pacc and ipsilateral thalamus , which showed slight reliability , and ( 2 ) contralateral pacc , s1 , and contralateral pins , which contained no clusters that were significantly reliable across the three sessions .
rois with the highest iccs ( the upper portion of the moderate range ) included amcc and s2 .
clusters of significantly reliable pain - related activation from selected rois are shown in fig . 
the results of the group - level contrast between significantly reliable pain - related activation for the 2 stimulus types are summarized in table  6 .
voxelwise icc values for each stimulus type were contrasted to obtain these results . the constant temperature stimulus ( 48  c ) produced more reliable activation than the constant perceived pain intensity stimuli in ains ( bilaterally ) , ipsilateral mpfc , and ipsilateral dlpfc ;
no significant differences in reliability were found for the other rois examined in this study .
thus , though there were some differences , no consistent pattern emerged in terms of whether a painful stimulus of constant temperature or of constant perceived intensity produced more reliable bold fmri responses .
clusters for which pain intensity ratings of the 48  c stimulus significantly predicted the magnitude of the bold response to the stimulus were found to be very limited , scattered , and with little overlap across sessions ( supplemental fig . 
clusters for which pain intensity ratings of the 48  c stimulus significantly predicted the magnitude of the bold response to the stimulus were found to be very limited , scattered , and with little overlap across sessions ( supplemental fig .  1 ) . as a result ,
as shown in supplemental tables  1 , 2 , and 3 , the effect of duration between sessions on the reliability of pain- and motor - related fmri activation was minor .
the iccs calculated in our original analysis and the analysis that took between - session duration into account are comparable and did not differ in a large or consistent way in any pain- or motor - related region of interest .
thus , the results suggest that with this data set , the duration between sessions did not contribute significantly to intersession variability .
as shown in supplemental tables  1 , 2 , and 3 , the effect of duration between sessions on the reliability of pain- and motor - related fmri activation was minor .
the iccs calculated in our original analysis and the analysis that took between - session duration into account are comparable and did not differ in a large or consistent way in any pain- or motor - related region of interest .
thus , the results suggest that with this data set , the duration between sessions did not contribute significantly to intersession variability .
significant motor - related activation was found in expected brain rois ( m1 and sma ) bilaterally .
intersession reliability of the spatial extent of significant motor - related activation is shown in table  7 .
the spatial reliability coefficient was high in m1 contralateral to the hand performing the motor task ( 0.78 ) but was much lower in contralateral sma ( 0.2 ) .
the spatial reliability coefficients in m1 and sma ipsilateral to the hand performing the motor task were notably lower than the contralateral regions .
intersession reliability of the amplitude of motor - related activation is shown in table  7 , which identifies the largest cluster of significantly reliable motor - related activation in each roi and the icc value associated with the peak voxel in the cluster ; additional clusters were also found in these rois .
substantial reliability ( based on shrout , 1998 ) was found in contralateral m1 ( icc  =  0.815 ) and moderate reliability in contralateral sma ( icc  =  0.657 ) .
fair reliability was found in m1 and sma ipsilateral to the hand performing the motor task .
overall , motor - related activation showed greater reliability than pain - related activation for both the spatial extent and icc measures .
this study quantitatively evaluated across - session reliability of pain - related bold fmri responses in brain areas that are typically regarded as part of the cortical pain network . while these brain areas show consistent and often robust activation in fmri studies across diverse pain modalities ( as reviewed in duerden and albanese , 2013 ) , the question of how reliable this activation is from session to session has not been comprehensively addressed in the literature .
we used two reliability measures ( voxelwise spatial overlap and iccs based on bold response amplitude ) that are commonly used in the literature ( bennett and miller , 2010 ) to assess test  retest reliability of fmri responses to a wide variety of conditions ( e.g. , motor tasks , visual stimulation , memory ) but have not been previously applied to pain .
the results revealed that the two measures of intersession pain - related fmri activation reliability used in this study produced disparate results , with ( 1 ) reliability based on spatial measures generally low and highly variable across brain regions and ( 2 ) reliability based on signal amplitude ( iccs ) in the fair - to - moderate range for most brain regions .
spatial reliability coefficients were low for most regions examined , indicating a low probability that the same voxels are activated across three separate sessions .
the highest spatial reliability coefficients were found in the ains for both stimulus types but no spatial overlap across the three sessions was found for pins . for both stimulus types , spatial reliability coefficients were higher in s2 than in s1 or thalamus , which is consistent with the findings of taylor and davis ( 2009 ) , who used a different measure of spatial reliability in a study involving mechanical pain applied across 4 sessions in 6 subjects ; their analysis was limited to s1 , s2 , and thalamus . while low s1 spatial reliability is in agreement with the inconsistency of s1 activation across fmri studies of pain ( bushnell et al . , 1999 ) , the reasons for a total lack of spatial overlap in pacc are unclear .
these results suggest that the precise location of pain - related activation is not highly reproducible across multiple sessions , perhaps due in part to variability introduced by motion correction , spatial normalization , and spatial smoothing procedures .
as noted by taylor and davis ( 2009 ) , conservative thresholding procedures such as those used in this study to avoid false positives may add to spatial variability across sessions by increasing false negatives ; for example , a voxel may be activated by pain in all sessions but be eliminated from one or more sessions if the activation falls slightly below threshold . as a result , spatial measures do not appear to be ideal for assessing intersession reliability because the threshold criterion has a major influence upon cluster sizes .
however , suprathreshold clusters of voxels in a brain region may show reliable pain - related bold signal amplitude changes across sessions . to test this possibility , iccs were calculated based on pain - related bold signal amplitude changes .
, 2009 ; mcgraw and wong , 1996 ; shrout and fleiss , 1979 ) .
this study used a conservative two - step approach that involved calculating iccs based on bold signal amplitude changes in suprathreshold voxels and then applying a filtering step to eliminate voxels with artifactually high iccs based on the variance structure of the data .
this approach revealed fair - to - moderate intersession reliability of pain - related activation in most regions of the cortical pain network , based on the conservative classification criteria described by shrout ( 1998 ) .
regional differences in intersession reliability were found , with the amcc ( bilaterally ) containing clusters with the highest iccs (   0.7 , with moderate reliability defined as 0.610.8 ) for both stimulus types .
other areas with iccs in the moderate range included the ains and most frontal lobe areas for both stimulus types , as well as s2 for the 50vas stimulus .
areas with the lowest intersession reliability based on the icc analysis also showed no to very low spatial reliability ; these regions included pacc , s1 , and pins for both stimulus types .
the finding of higher iccs in s2 than s1 is consistent with taylor and davis ( 2009 ) , though their calculated iccs were higher than those in this study , likely because they included responses from subthreshold voxels in their analysis .
the brain regions that displayed the highest intersession reliability in this study were ains ( based on both spatial extent and icc measures ) and amcc ( based on iccs ) .
both of these areas have been implicated in processing affective aspects of pain , and are anatomically and functionally connected ( berthier et al . , 1988 ; friedman et al . , 1986 ; heimer and van hoesen , 2006 ; vogt , 2005 ) .
craig ( 2002 , 2003 , 2009 , 2011 ) has postulated that the ains has a role in generating subjective emotional feelings about the internal state of the body and interacts with the acc , which initiates adaptive behavioral responses .
evidence for the involvement of amcc in emotion - based response selection , including fear avoidance behavior ( vogt , 2005 ) , provides support for craig s model .
thus , the consistently reliable activation of ains and amcc in this study may highlight the importance of and priority given to processing emotional - motivational aspects of pain .
however , craig s model identifies pins as the region that generates the initial cortical representation of the body s homeostatic condition , providing ains with information upon which to generate emotions associated with that representation .
the fact that the pins had very low intersession reliability in this study does not fit well with a model of a serial - processing insular system in which the anterior region response is highly dependent upon the posterior region response .
the relatively high iccs and spatial overlap for contralateral s2 contrasts sharply with that of the pins .
both regions have historically been considered important in nociceptive processing , and most functional neuroimaging studies do not describe differences in their nociceptive processing capacities . the current study reveals the much more reliable responsiveness of the s2 cortex to acute heat stimuli , compared to the pins .
this difference suggests that s2 has a more essential role in nociceptive processing , at least within the context of responding to repetitively administered acute noxious heat stimuli .
the second objective of this study was to evaluate whether bold fmri activation produced by painful heat stimuli of constant temperature or of constant perceived pain intensity was more reliable . as expected , spatial overlap of significant pain - related activation between these two conditions was relatively high , indicating that the experience of heat pain , regardless of these differences in intensities , activates common brain areas .
this is consistent with a recent report that pain - related fmri activation did not differ for fixed temperature stimuli and perceptually equalized stimuli in any brain region , including the pacc , amcc , and ins ( van den bosch et al .
the icc contrast between the two stimulus conditions showed only a few brain regions where reliability differed ( ains , s2 , and some frontal lobe regions ) . among these brain regions ,
no clear pattern emerged : in some areas the constant temperature stimulus was more reliable across sessions while the opposite was found for other areas .
thus , in this paradigm , no definitive conclusion can be made regarding intersession reliability differences in fmri activation resulting from stimuli of constant temperature versus stimuli that produce a constant perception .
one possible explanation is that variation in perceived pain intensity is not an important contributor to the variability of the bold response , a notion that is supported by our finding of only scattered clusters for which pain intensity ratings significantly covaried with the bold signal amplitude response to painful stimuli .
another possible explanation for failing to find a consistent difference is that the methodology of the current study ( such as variability in the subject pool or choice of stimuli ) did not provide the range of data adequate to draw out differences .
thus , the possibility remains that in some populations or contexts , pain - related fmri activation may be more reliable for stimuli of fixed temperature or fixed perception ; studies that further explore this issue may provide valuable information to aid in the design of pain imaging experiments .
a major strength of this study is that the analytical approach used to assess reliability of pain - related activation was also applied to fmri data collected from the same subjects during the same sessions as they performed a simple motor task ( finger - thumb opposition ) . using this approach , intersession reliability of motor - related activation in this study
was found to be comparable to previously published results for both spatial overlap and icc measures ( bennett and miller , 2010 ; gountouna et al .
, 2006 ; kong et al . , 2007 ; mcgregor et al . , 2012 ; yoo et al . , 2005 ) .
this supports the validity and appropriateness of the analytical approach used in this study to assess intersession reliability of pain - related fmri activation .
this study is the first to quantitatively examine intersession reliability of pain - related fmri activation in the entire cortical pain network , expanding upon the work of taylor and davis ( 2009 ) , who limited their analysis to somatosensory processing regions .
furthermore , this study is the first to examine reliability of activation evoked by a ramp - and - hold contact heat stimulus paradigm , which is used in many pain imaging paradigms , including those assessing changes associated with pain - reducing manipulations .
while the sample size of 14 subjects in this study is modest , it is the largest used to evaluate intersession reliability of pain - related fmri activation and is greater than the average number of 11 subjects across all test  retest fmri activation studies ( bennett and miller , 2010 ) .
furthermore , the diversity of our subjects in terms of age and gender , which likely added to inter - individual variability of brain responses , can be viewed as a strength , as the results should be more generalizable than those obtained from a more homogeneous subject population such as undergraduate students , as utilized in many test 
habituation is a possible explanation for finding poor intersession pain - related fmri reliability in some brain areas .
however , consistent with published literature for the type of ramp - and - hold contact heat stimulus paradigm used in this study ( quiton and greenspan , 2008 ) , pain intensity ratings did not change across the three sessions and the temperatures required to produce subject - specific ratings of 50 on a 0100 scale did not change over the course of the experiments .
furthermore , the data show no systematic decreases in either number of significant voxels or signal amplitude changes across the three sessions .
2008 ) required daily application of 60 painful heat stimuli to the forearm for 8  days before observing perceptual habituation and decreases in pain - related fmri responses in the brain .
thus , it is unlikely that habituation is a major contributor to the poor intersession reliability observed for some brain areas in this study .
conclusions from this study , in which 12 trials of each temperature were applied in each session , may not be generalizable to studies where the number of trials is significantly different .
increasing the number of stimuli has the potential to increase statistical power and intersession reliability ; however , it also might introduce dynamic changes in the bold response that would add variability and thereby decrease intersession reliability .
a separate study is needed to determine how reliability varies as a function of stimulus number .
overall , in this paradigm , pain - related bold fmri responses showed fair to moderate test 
retest reliability in brain regions that are part of the classical pain network . a review of over 63 studies of fmri test 
retest reliability for various tasks , designs , and test  retest intervals reported the average icc value was 0.5 ( bennett and miller , 2010 ) .
the iccs calculated in this study for most brain regions were greater than 0.5 , suggesting that pain - related fmri activation has better than average intersession reliability .
the finding that some brain regions showed stronger test  retest reliability than others may provide useful information to guide longitudinal pain studies .
in addition , the study findings led to the following recommendations for test  retest reliability analyses : ( 1 ) spatial extent and localization of activation do not appear to be useful measures of fmri response reliability at the group level .
( 2 ) the use of icc values alone as a measure of reliability may not be sufficient , as the underlying variance structure of a dataset may result in erroneously high icc values .
methods to eliminate erroneously high icc values ( such as the filtering step used in this study ) should be incorporated into any analysis of this type .
the following are the supplementary data related to this article.supplemental figure  1voxels for which pain intensity ratings of a 48c stimulus significantly covaried with the bold response to the stimulus in ( a ) session 1 ,
( b ) session 2 , and ( c ) session 3.supplemental table  1brain regions with significantly reliable pain - related signal amplitude associated with 48c stimulus using between session duration as a covariateresults of original analysis ( as presented in table 4)supplemental table  2brain regions with significantly reliable pain - related signal amplitude associated with 50vas * stimulus using between session duration as a covariateresults using between session duration as a covariateoriginal results ( as reported in table 5)supplemental table  3reliability measures for motor task activation using between session duration as a covariate brain regions with significantly reliable motor task - related signal amplitude using between session duration as a covariateresults of original analysis ( as reported in table 7 ) voxels for which pain intensity ratings of a 48c stimulus significantly covaried with the bold response to the stimulus in ( a ) session 1 ,
brain regions with significantly reliable pain - related signal amplitude associated with 48c stimulus using between session duration as a covariate results of original analysis ( as presented in table 4 ) brain regions with significantly reliable pain - related signal amplitude associated with 50vas * stimulus using between session duration as a covariate results using between session duration as a covariate original results ( as reported in table 5 ) reliability measures for motor task activation using between session duration as a covariate brain regions with significantly reliable motor task - related signal amplitude using between session duration as a covariate results of original analysis ( as reported in table 7 ) supplementary data to this article can be found online at http://dx.doi.org/10.1016/j.nicl.2014.07.005 .
 as the practice of conducting longitudinal fmri studies to assess mechanisms of pain - reducing interventions becomes more common , there is a great need to assess the test  retest reliability of the pain - related bold fmri signal across repeated sessions . 
 this study quantitatively evaluated the reliability of heat pain - related bold fmri brain responses in healthy volunteers across 3 sessions conducted on separate days using two measures : ( 1 ) intraclass correlation coefficients ( icc ) calculated based on signal amplitude and ( 2 ) spatial overlap . 
 the icc analysis of pain - related bold fmri responses showed fair - to - moderate intersession reliability in brain areas regarded as part of the cortical pain network . 
 areas with the highest intersession reliability based on the icc analysis included the anterior midcingulate cortex , anterior insula , and second somatosensory cortex . 
 areas with the lowest intersession reliability based on the icc analysis also showed low spatial reliability ; these regions included pregenual anterior cingulate cortex , primary somatosensory cortex , and posterior insula . 
 thus , this study found regional differences in pain - related bold fmri response reliability , which may provide useful information to guide longitudinal pain studies . 
 a simple motor task ( finger - thumb opposition ) was performed by the same subjects in the same sessions as the painful heat stimuli were delivered . 
 intersession reliability of fmri activation in cortical motor areas was comparable to previously published findings for both spatial overlap and icc measures , providing support for the validity of the analytical approach used to assess intersession reliability of pain - related fmri activation . 
 a secondary finding of this study is that the use of standard icc alone as a measure of reliability may not be sufficient , as the underlying variance structure of an fmri dataset can result in inappropriately high icc values ; a method to eliminate these false positive results was used in this study and is recommended for future studies of test  retest reliability . 
Introduction
Methods
Results
Discussion