the patient data analyzed in this study were obtained from the united kingdom glaucoma treatment study ( ukgts ) .
the ukgts was a randomized , double - masked , placebo - controlled , multicenter treatment trial for open - angle glaucoma .
the study recruited newly diagnosed ( untreated ) glaucoma cases with glaucomatous visual field defects consistent with optic nerve head changes and with open angles on gonioscopy .
subjects with moderately advanced visual field loss ( mean deviation worse than 10 db in the better eye or worse than 16 db in the other eye ) were excluded from the study and subjects with iop greater than 35 mm hg on two consecutive visits were also excluded from the study .
subjects with lens opacity greater than p1 ( on the lens opacity classification system iii grading ) and those with other ocular comorbidities such as diabetic retinopathy were also excluded from the study .
further details on the inclusion and exclusion and other types of tests included in the ukgts study can be found elsewhere .
we analyzed test results for perimetry , neuroretinal rim area , and circumpapillary rnfl thickness .
the visual field sensitivity measures were obtained using the goldmann size iii stimulus on the humphrey field analyzer ( hfa ) ii or ii - i , ( carl zeiss meditec , dublin , ca , usa ) with the sita 24 - 2 program .
humphrey field analyzer test results with fixation losses greater than 20% or false - positive rate greater than 15% were excluded .
the exclusion criteria did not exempt any subjects in the original ukgts dataset from our study .
the rim area measurements were obtained using the hrt-3 ( software version 3.0.60 ; heidelberg engineering , heidelberg , germany ) . when the mean pixel sd of the hrt image was greater than 40 m , the hrt data were regarded as unreliable and were excluded .
rim area data were collected on 482 right eyes and 481 left eyes . after applying the exclusion criteria
we were left with hrt data for 482 and 480 right and left eyes , respectively .
the rnfl thicknesses were acquired using the stratus oct ( software version 5.0 ; carl zeiss meditec ) .
the fast rnfl thickness scanning protocol was used ( 3.4-mm diameter circle centered on the disc ) .
the oct data were screened to exclude scans with signal strength less than 7 . for our analysis
we also excluded oct scans with errors : with an error message , a pixel lower than 10 m , or the range across repeated scans greater than 15 m .
there were oct data on 289 right eyes and 277 left eyes in the original ukgts data set .
after applying our inclusion and exclusion criteria , we were left with rnfl thickness measures for 284 and 261 right and left eyes , respectively .
we matched the hrt , oct , and hfa data by date and eye for the last visit of each patient .
we required that all three tests were acquired on the same day , in order to avoid artifacts from progression .
we were able to successfully match the three tests for 245 right eyes and 223 left eyes .
the 55 controls were from a pool of 62 whose oct and hfa data had already been published in swanson et al . only subjects who also had reliable data on the hrt
the criteria for excluding unreliable data and matching reliable data were the same for the controls as for the patients .
the instruments used in acquiring the control data were comparable with those used in the ukgts study . in the original studies
it was reported that the methods used in gathering the data were in accordance with the declaration of helsinki .
informed consent was obtained from subjects after explanation of the nature and goals of the study , before testing began . in our primary analysis , we analyzed data from the right eyes in an effort to replicate the finding that between - subject variability in controls is the primary source of structural  functional ( and structural  structural ) discordance in patients with glaucoma .
we analyzed data from the left eyes in a secondary confirming analysis , in the same way as for the right eye .
global measures from hfa , oct , and hrt were used to reduce the impact of between - subject variations in nerve fiber projection to optic disc sectors . for the hfa ,
global visual field sensitivity was calculated by first converting the sensitivity at the various test locations to a linear scale before averaging .
two visual field points , at and just above the blind spot , were excluded from this index . in order to overcome the impact of the differences in the metric of measurement of the various clinical measuring techniques on our analysis , we computed depth of defect ( difference from mean normal ) for data from each device .
depth of defect was used as a measure of the severity of the glaucomatous damage .
our analysis was centered around the limits of agreement between two measures as described by bland and altman ; the difference in depth of defect for two tests was plotted against their average depth of defect .
the difference in defect depth may vary with the severity of damage , so we calculated the residuals from linear regression of difference versus mean .
the absolute values of the residuals were plotted against the average of the two measures and then fitted with a regression to estimate whether the limits of agreement varied with severity of damage . in our analysis
, the limits of agreement did not vary with the average depth of defect and so we did not proceed with the other methods described in section 3.3 of the bland and altman article ( that describe computing limits of agreement that vary with the average depth of defect ) .
functional comparisons ( perimetry versus rnfl thickness , perimetry versus rim area ) and structural  structural comparisons ( rnfl thickness versus rim area ) .
it has been found , that relative to the hfa , depth of defect estimated from structural measures is limited by a floor ( due to the nonneural component contained in the structural measures ) .
this floor introduces a potential artifact to the bland - altman analysis when two measures ( containing different amounts of nonneural components ) are being compared .
a floor of 0.5 was used in swanson et al . in the bland - altman plot comparing hfa and oct ( and oct and hrt ) we also used a floor of 0.5 . for lack of a study that established a floor for hrt measures and for the sake of simplicity we also used a floor of 0.5 in the hfa - hrt comparison . in the primary analysis ,
statistical significance was set to a p value of 0.017 ( a bonferroni adjustment ) because three f values ( two structural  functional comparisons and one structural  structural comparison ) were assessed . for a p value of 0.017 and the sample sizes in the patient and control groups ,
for these sample sizes , the critical value will be f(244 , 54 ) = 1.63 , p less than 0.017 . to compare our findings with those of hood et al .
, we applied the hood - kardon model ( which predicts the relationship between rnfl thickness and perimetry ) to our data .
the hood - kardon model assumes that the nonneural component of the rnfl thickness contributes approximately 33% of the measured thickness before the onset of the glaucomatous damage .
it also assumes that rnfl thickness does not increase beyond mean normal even when visual field sensitivity is above mean normal . to allow for a direct comparison with our analyses we made similar assumptions as hood et al .
( although our analysis was focused on global indexes and that for hood et al . was for superior temporal and inferior temporal disc sectors ) .
the model predicts that approximately 95% of the patient data points should be within the prediction interval based on variability in the control data .
the patient data analyzed in this study were obtained from the united kingdom glaucoma treatment study ( ukgts ) .
the ukgts was a randomized , double - masked , placebo - controlled , multicenter treatment trial for open - angle glaucoma .
the study recruited newly diagnosed ( untreated ) glaucoma cases with glaucomatous visual field defects consistent with optic nerve head changes and with open angles on gonioscopy .
subjects with moderately advanced visual field loss ( mean deviation worse than 10 db in the better eye or worse than 16 db in the other eye ) were excluded from the study and subjects with iop greater than 35 mm hg on two consecutive visits were also excluded from the study .
subjects with lens opacity greater than p1 ( on the lens opacity classification system iii grading ) and those with other ocular comorbidities such as diabetic retinopathy were also excluded from the study .
further details on the inclusion and exclusion and other types of tests included in the ukgts study can be found elsewhere .
we analyzed test results for perimetry , neuroretinal rim area , and circumpapillary rnfl thickness .
the visual field sensitivity measures were obtained using the goldmann size iii stimulus on the humphrey field analyzer ( hfa ) ii or ii - i , ( carl zeiss meditec , dublin , ca , usa ) with the sita 24 - 2 program .
humphrey field analyzer test results with fixation losses greater than 20% or false - positive rate greater than 15% were excluded .
the exclusion criteria did not exempt any subjects in the original ukgts dataset from our study .
the rim area measurements were obtained using the hrt-3 ( software version 3.0.60 ; heidelberg engineering , heidelberg , germany ) . when the mean pixel sd of the hrt image was greater than 40 m , the hrt data were regarded as unreliable and were excluded .
rim area data were collected on 482 right eyes and 481 left eyes . after applying the exclusion criteria
we were left with hrt data for 482 and 480 right and left eyes , respectively .
the rnfl thicknesses were acquired using the stratus oct ( software version 5.0 ; carl zeiss meditec ) .
the fast rnfl thickness scanning protocol was used ( 3.4-mm diameter circle centered on the disc ) .
the oct data were screened to exclude scans with signal strength less than 7 . for our analysis
we also excluded oct scans with errors : with an error message , a pixel lower than 10 m , or the range across repeated scans greater than 15 m .
there were oct data on 289 right eyes and 277 left eyes in the original ukgts data set .
after applying our inclusion and exclusion criteria , we were left with rnfl thickness measures for 284 and 261 right and left eyes , respectively .
we matched the hrt , oct , and hfa data by date and eye for the last visit of each patient .
we required that all three tests were acquired on the same day , in order to avoid artifacts from progression .
we were able to successfully match the three tests for 245 right eyes and 223 left eyes .
the 55 controls were from a pool of 62 whose oct and hfa data had already been published in swanson et al . only subjects who also had reliable data on the hrt
the criteria for excluding unreliable data and matching reliable data were the same for the controls as for the patients .
the instruments used in acquiring the control data were comparable with those used in the ukgts study . in the original studies
it was reported that the methods used in gathering the data were in accordance with the declaration of helsinki .
informed consent was obtained from subjects after explanation of the nature and goals of the study , before testing began .
in our primary analysis , we analyzed data from the right eyes in an effort to replicate the finding that between - subject variability in controls is the primary source of structural  functional ( and structural  structural ) discordance in patients with glaucoma .
we analyzed data from the left eyes in a secondary confirming analysis , in the same way as for the right eye .
global measures from hfa , oct , and hrt were used to reduce the impact of between - subject variations in nerve fiber projection to optic disc sectors . for the hfa ,
global visual field sensitivity was calculated by first converting the sensitivity at the various test locations to a linear scale before averaging .
two visual field points , at and just above the blind spot , were excluded from this index . in order to overcome the impact of the differences in the metric of measurement of the various clinical measuring techniques on our analysis , we computed depth of defect ( difference from mean normal ) for data from each device .
depth of defect was used as a measure of the severity of the glaucomatous damage .
our analysis was centered around the limits of agreement between two measures as described by bland and altman ; the difference in depth of defect for two tests was plotted against their average depth of defect .
the difference in defect depth may vary with the severity of damage , so we calculated the residuals from linear regression of difference versus mean .
the absolute values of the residuals were plotted against the average of the two measures and then fitted with a regression to estimate whether the limits of agreement varied with severity of damage . in our analysis , the limits of agreement did not vary with the average depth of defect and so we did not proceed with the other methods described in section 3.3 of the bland and altman article ( that describe computing limits of agreement that vary with the average depth of defect ) .
functional comparisons ( perimetry versus rnfl thickness , perimetry versus rim area ) and structural  structural comparisons ( rnfl thickness versus rim area ) .
it has been found , that relative to the hfa , depth of defect estimated from structural measures is limited by a floor ( due to the nonneural component contained in the structural measures ) .
this floor introduces a potential artifact to the bland - altman analysis when two measures ( containing different amounts of nonneural components ) are being compared .
a floor of 0.5 was used in swanson et al . in the bland - altman plot comparing hfa and oct ( and oct and hrt ) we also used a floor of 0.5 . for lack of a study that established a floor for hrt measures and for the sake of simplicity
we also used a floor of 0.5 in the hfa - hrt comparison . in the primary analysis ,
statistical significance was set to a p value of 0.017 ( a bonferroni adjustment ) because three f values ( two structural  functional comparisons and one structural  structural comparison ) were assessed . for a p value of 0.017 and the sample sizes in the patient and control groups ,
for these sample sizes , the critical value will be f(244 , 54 ) = 1.63 , p less than 0.017 . to compare our findings with those of hood et al .
, we applied the hood - kardon model ( which predicts the relationship between rnfl thickness and perimetry ) to our data .
the hood - kardon model assumes that the nonneural component of the rnfl thickness contributes approximately 33% of the measured thickness before the onset of the glaucomatous damage .
it also assumes that rnfl thickness does not increase beyond mean normal even when visual field sensitivity is above mean normal . to allow for a direct comparison with our analyses we made similar assumptions as hood et al .
( although our analysis was focused on global indexes and that for hood et al . was for superior temporal and inferior temporal disc sectors ) .
the model predicts that approximately 95% of the patient data points should be within the prediction interval based on variability in the control data .
for 245 right eyes and 223 left eyes , the patients had mean ( sd ) age of 65.5 ( 11.1 ) and 65.6 ( 11.2 ) years , respectively .
the 55 controls had mean ( sd ) age of 63.0 ( 9.9 ) years .
for the controls , mean ( sd ) perimetric sensitivity was 0.49 ( 0.14 ) log contrast sensitivity , mean rnfl thickness was 1.99 ( 0.04 ) log m and mean rim area 0.18 ( 0.04 ) log mm .
the mean control hrt disc area was 1.93 ( 0.41 ) mm and that for the patients was 2.02 ( 0.40 ) mm and 2.00 ( 0.47 ) mm for od and os , respectively .
the table shows the sds of the residuals when depths of defect measured by two devices were compared on a bland - altman plot .
the sd in log units of the residuals around the regression line on a bland - altman plot comparing two clinical device outputs in patients the left panels in figure 1 show plots of discordance against average depth of defect comparing perimetry and rnfl thickness
. it also features the hood - kardon model modified for the plot . in the right and left eyes , 4.7% and 5.4% , respectively , of the patient data fell outside the 95% limits based on the dispersion in the control population .
the upper plots show data for left eyes and the lower plots show data for right eyes .
the left panels show limits comparable to the hood - kardon model , and the right panels show the 95% limits of agreement comparable to the limits of swanson et al .
the red ellipses on the left show the 95% prediction interval for the controls ; the black broken lines show the 95% prediction interval for patients .
the slope of the regression line on the bland - altman plot was 0.87 ( se = 0.07 ) and 0.88 ( se = 0.06 ) for right and left eyes , respectively ; the intercepts of the regression lines were 0.07 ( se = 0.01 ) and 0.06 ( se 0.01 ) for left and right eyes respectively .
a second structural  functional comparison is shown in figure 2 , perimetry versus rim area .
the slope of the regression line was 0.44 ( se = 0.10 ) for right eyes and 0.59 ( se = 0.10 ) for left eyes ; the intercepts were 0.13 ( se = 0.02 ) and 0.10 ( se = 0.02 ) for the right and left eyes , respectively .
the upper plot shows data for left eyes and the lower plot shows data for right eyes .
a structural  structural comparison is shown in figure 3 , rim area versus rnfl thickness .
the slope of the regression line on the bland - altman plot was 0.56 ( se = 0.07 ) for right eyes and 0.47 ( se = 0.08 ) for left eyes ; the intercept was 0.00 ( se = 0.01 ) for both right and left eyes . a bland - altman plot comparing the rnfl thickness and rim area . as in figures 1 and 2 ,
the upper plot shows data for left eyes and the lower plot shows data for right eyes .
in this study , we replicate the finding that normal between - subject variability is the primary source of structural  functional discordance in patients with glaucoma .
we compared the variability of the discordance in the patients with the variability of the discordance in the controls and found them to be similar .
we extended the analysis to include a structural  structural comparison and found a similar result ; variability in the control group was similar to variability in the patient group .
we also analyzed the perimetry versus rnfl thickness data using an extension of the hood - kardon model , and found that approximately 95% of the patient data fell within the 95% prediction interval computed from the control population . in the hfa - hrt comparison
however , we noticed a relatively larger variance as compared with the hfa - oct and oct - hrt comparisons .
we were therefore interested in investigating the impact of disc area on the hfa - hrt comparison using the moorfields regression analysis . compared with our original analysis , adjusting for the rim area ( using disc area and age ) in the hfa - hrt comparison
increased the variance in the right of patients by 1% and decreased the variability in the control group by 3% .
overall , adjusting for the rim area ( using the disc size and age ) did not result in a significant improvement in the hfa - hrt comparison .
this is not however surprising because we did not see any consistent distribution in depth of defect based on disc size on our plots .
this increased variability may be due to some image acquisition factors like the operator tracing the area of the disc and the limit imposed on the rim area by the disc size . in the comparison of perimetry and rnfl thickness , the regression on the bland - altman plot yielded a positive slope , meaning that as the average defect depth increased , perimetry tended to give increasingly deeper defects than did rnfl thickness .
the comparison of perimetry and rim area also found positive slopes , but they were not as steep as for perimetry versus rnfl thickness .
this is consistent with the proposal by shafi et al . of a greater nonneural component for rnfl thickness than for rim area .
the direct comparison of rim area and rnfl thickness in figure 3 also supports a greater nonneural component for rnfl thickness than for rim area . we recomputed the discordance between the rim area and rnfl thickness measures using a series of other floors to investigate whether the finding on the relative amount of nonneural component contained in the rim area and
rnfl thickness measures was impacted by our choice of floor ( 0.5 log unit ) .
we found that although the magnitude of slope of the bland - altman plot changed depending on the floor , its ( negative ) direction and implication did not change .
the intercept of the bland - altman regression line was nonzero for perimetry versus rnfl thickness and for perimetry versus rim area , but not for rim area versus rnfl thickness .
this is an indication that at the early onset of glaucomatous damage , structural measures estimate deeper defects than perimetry .
a recent study from our research group found a similar result when comparing sap with new forms of perimetry resistant to peripheral defocus : nonzero intercepts for bland - altman regression line indicating that sap defects were not as deep in mild defects and became deeper in more severe defects .
hood et al . pointed out that when two tests are being compared , the test with a smaller sd in the control population would require less damage to reach statistical significance and be flagged as glaucomatous than the test with a larger sd .
drawing from the statistics of identifying which loss constitutes glaucomatous damage and which loss does not , the oct is likely to be more sensitive than the hfa because it had a smaller sd in the control group than the hfa . in an exploratory analysis
, we calculated the fifth percentile for the rnfl thickness and visual field measures from the control population .
we set the value computed as threshold and identified subjects as showing glaucomatous damage when their rnfl thickness ( or visual field sensitivity ) fell below the threshold .
the oct identified more subjects than the hfa as showing glaucomatous damage consistent with the finding of hood et al .
, this may explain why the oct appears to be more sensitive to early damage and suggests a different reasoning ( as to why the oct may flag mild defects earlier than the hfa ) than  preperimetric  glaucoma concept ( which suggests that a large percentage ofganglion cells are lost before the loss is detected on perimetric testing ) .
from the comparisons , we also noticed that the limits of agreement of oct - hfa comparison ( a structural  functional comparison ) in the patient group were similar to the limits of agreement of the oct - hrt comparison ( a structural  structural comparison ) .
this provides further evidence that the discordance seen in comparing structural measures to functional measures may not be necessarily due to the fact that one method assesses glaucomatous damage through structural loss while the other uses functional loss .
we investigated the characteristics of disc parameters ( such as cup - to - disc volume , cup volume , cup  disc ratio , etc . ) reported by the hrt in the control subjects with the most extreme discordance for each comparison to determine whether there are any specific structural patterns reported by the hrt that may explain the variability of the discordance in controls .
for the hfa - oct comparison we did not find any significant difference between subjects with the most extreme positive discordance ( greater hfa depth of defect than oct ) and those with the most extreme negative discordance ( greater oct depth of defect than hfa ) .
for example , the mean disc area of controls with the highest 10% positive discordance ( greatest hfa depth of defect than oct ) and those with the highest 10% negative discordance ( greatest oct depth of defect than hfa ) were 1.99 mm ( sd = 0.15 ) and 1.82 mm ( sd = 0.30 ) .
however , in the hrt - oct comparison we noticed that the maximum cup depth , mean cup depth , horizontal and vertical cup - to - disc area ratio were significantly higher ( at p < 0.05 ) in subjects with the most extreme positive discordance ( greater hrt depth of defect than oct ) than those with the most extreme negative discordance ( greater oct depth of defect than hrt ) . in a consistent manner , the rim disc area ratio and rim volume were significantly smaller in subjects with the most extreme positive discordance than subjects with the most extreme negative discordance .
thus , adjusting for maximum cup depth , mean cup depth , or horizontal and vertical cup - to - disc ratio may reduce the discordance in the healthy population for oct - hrt comparison .
we also noticed that the majority of subjects with the most extreme discordance in the hfa - oct comparison were not the same as the subjects with the most extreme discordance in the hfa - hrt comparison .
thus , the few subjects who have extreme discordance on both the hfa - oct and hrt - oct comparison may have some underlying structural pattern common to both the hrt and oct that causes the structural depth of defect to differ from the functional depth of defect in a consistent manner .
however , it is not clear whether the subjects who have higher depth of defect on the hfa - oct comparison but not on the hfa - hrt measure or vice versa have any specific structural pattern .
the trend in this group is consistent with the hrt - oct discordance and suggests a multidimensional source of structural  functional discordance .
thus , the discordance observed in a subject is not only a property of the structural and functional ganglion cell integrity but is also influenced by the region on the retina where the structural integrity is being assessed . in conclusion , we confirmed the finding of prior studies that between - subject variability in healthy eyes is the primary source of structural  functional discordance in patients , and extended this to structural  structural comparisons . developing techniques with reduced between - subject variability in the healthy population may be helpful in improving the discordance in comparing two techniques . there have been proposals on how to reduce between - subject variability in controls .
patel et al . proposed using rnfl volume instead of rnfl thickness in structural measures ; elsewhere , there are proposals on how to reduce the impact of prereceptoral factors on perimetric sensitivity .
these proposals are geared toward reducing the normal between - subject variability and may consequently yield more sensitive measures with reduced structural  functional ( or structural  structural ) discordance when two measures are compared .
improved techniques for structural and functional measures yielding improved structural  functional discordance may not only be necessary to complement a clinician 's diagnosis but will also open discussions for the use of perimetry for testing ganglion cell dysfunction . to address the challenge of structural  functional discordance holistically , other factors that need to be addressed include the spatial sampling of perimetric measures and the difficulty in spatially mapping structural abnormality to functional abnormality using the structural  functional maps . while the structural measures ( acquired in the region of the disc ) assess all the ganglion cell axons present in the retina , perimetric testing with the 24 - 2 protocol samples 54 testing locations on the retina .
this predisposes the perimetric tests to miss localized wedge defects that may fall between perimetric testing locations while being identified by the structural measures . due to the fact that quite a number of the structural measures concentrated on determining glaucomatous structural abnormality at the region of the disc , most studies are faced with the challenge of choosing a structural  functional map to be able to relate structural abnormality to functional abnormality .
while our study concentrated on using global measures to overcome this challenge , it is clear that the wide range of variability in the trajectory of the retinal nerve fiber bundles to the disc is a potential source of structural  functional discordance .
ballae ganeshrao et al . have demonstrated that accurately mapping structural measures to functional measures spatially accounting for the anatomical differences in these trajectories yield improved structural 
the developments in imaging techniques that allow the visualization of the nerve fiber bundles en face are an attractive approach to overcoming the spatial challenges implicated in the structural  functional discordance and may provide more insight into the true nature of the structural - functional relationship .
 purposeto test with an independent data set the finding that between - subject variability in healthy eyes is the primary source of structural  functional discordance in patients with glaucoma.methodsneuroretinal rim area , retinal nerve fiber layer thickness , and perimetric data were analyzed for one eye in each of 55 control subjects and for 245 right eyes of patients in the united kingdom glaucoma treatment study . 
 data were gathered with the heidelberg retina tomograph ( hrt ) , stratus optical coherence tomograph ( oct ) , and humphrey field analyzer ( hfa ) . 
 discordance was quantified as width of the limits of agreement from a bland - altman analysis of depth of defect . 
 the ratio of variances ( f test ) for the patient and control groups was computed for comparisons of hfa - oct , hfa - hrt , and oct - hrt . 
 bonferroni adjustment required p less than 0.017 for statistical significance . 
 the discordance in the patients was also quantified as the 95% prediction interval computed from the discordance in controls using the hood - kardon model for the hfa - oct comparison.resultsthe f ratio comparing discordance in patients and controls was 0.77 , 1.43 , and 1.32 for the hfa - oct , hfa - hrt , and oct - hrt comparisons with p values 0.88 , 0.06 , and 0.11 , respectively . for the hood - kardon model , 
 4.7% of the patients had discordance outside the 95% prediction interval computed from the discordance in controls . 
 similar results were obtained when all comparisons were repeated for left eyes of patients.conclusionsthese results confirm previous findings that between - subject variability in healthy eyes is the primary source of structural  functional discordance in patients with glaucoma , and extends this finding to a structural  structural comparison . 
Methods
Subjects and Devices
Analysis
Results
Discussion