extremely low birth weight infants ( elbw ) pose a heavy resource burden in perinatal healthcare . while it is inevitable that elbw infants would have relatively long neonatal intensive care unit ( nicu ) length of stay ( los ) , there may be opportunities for improving and standardizing care , as unexplained variation in los has been described for premature infants .
depending on the patient , hospital , and payer , there may be financial considerations that could influence practice in such a manner to discharge such infants sooner or later .
for these reasons , after baseline characteristics that contribute meaningful increments in explanatory power are accounted for , los may be considered a quality of care measurement . in order to use elbw los as a quality measurement amongst hospitals ,
we need to reveal the impact of practice variation among providers via a model employing inclusion / exclusion criteria and baseline patient characteristics that are broadlyaccepted by relevant stakeholders .
previous studies on modeling length of stay for premature infants have been limited due to being confined to one or a few centers or concerned only certain level nicus . a population - based study in sweden evaluated moderately preterm infants 3034 weeks .
a study of 30 nicus in the uk included a broad group of premature infants including those 23 to 32 weeks gestation and with birth weight up to 3000 grams .
our aim was to characterize los for the smallest , most vulnerable infants who draw especially heavily on nicu resources , across wide levels of hospital care .
the purpose of this study was to modellos for elbw infants in order to examine provider variation and thus to inform quality assessmentwithin a population - based dataset .
because both widely heterogeneous patient care needs and patient care in more than one discrete system of care can obscure the true contribution of various independent risk factors to los , we studied elbws who survived and were cared for from birth or shortly after birth at a single institution until discharge to home .
in particular , we considered : 1 ) type of risk adjustment model , 2 ) the potential for bias from excluding patients who died before discharge , and 3 ) the potential for bias from excluding patients who were transferred to a lower level hospital for convalescent care .
we used data from the california perinatal quality care collaborative ( cpqcc ) , a voluntary collective of 129 nicus that report demographic and clinical data into a central repository .
more than 90% of nicus in california participated in this collaborative between january 1 , 2008 , and december 31 , 2010 .
member nicus collect data on their patients in a prospective manner , and each record undergoes a variety of range , logic , and missing data checks .
the period of analysis included infants cared for in cpqcc nicus for the three years , 2008 to 2010 .
inclusion criteria were infants with birth weight 4011,000 grams who were cared for at a cpqcc hospital from birth or within the first day after delivery , until discharge to home .
exclusion criteria were patients who died , those with congenital anomalies , and infants who had any surgery except circumcision .
the primary outcome ofinterest was length of stay ( los ) , defined as the number of days from admission to discharge . for infants
those infants not cared for at a cpqcc hospital on the first day , the first day of hospital stay at a cpqcc hospital counted as day one .
the following baseline characteristics were considered in risk adjustment : infant characteristics ( sex , birth weight , gestational age , small for gestational age ( sga ) ) , maternal and intrapartum variables ( maternal age , race / ethnicity , prenatal care , location of birth ( inborn / outborn ) , antenatal steroid use , multiple birth , mode of delivery , antenatal conditions ) , and delivery room variables ( 5 minute apgar scores ) . in considering sga (
defined as birth weight < 10th percentile for gestational age ) , it was noted that sga infants had shorter los , particularly those with gestational age > = 30 weeks . as birth weight
< 1,000 grams was an inclusion criterion , we used an approach that accounted for the birth weight restriction of the eligible cohort .
the sga variable was categorized into 2 different groups : 1 ) sga infants < 30 weeks of gestational age , and 2 ) sga infants > = 30 weeks ) .
we chose 30 weeks as the cut point because the 3rd percentile of 1,000 gram infants is approximately 30 weeks .
the rationale was to distinguish small size by infants developmental stage in terms of gestational age .
previous research has shown a survival advantage for sga neonates of < 31 weeks associated with cesarean delivery .
we examined collinearity among the model variables under consideration , as collinearity may influence estimation and inference for the primary parameters and lead to a loss of power , considering variance inflation factors and tolerance ( see appendix and table e1 for details on this selection process ) . from this analysis
, we included birth weight and sga , but not gestational age as explanatory variables in models .
skewness in the empirical distribution with a long positive side tail suggested that the use of the non - parametric test ( kruskal - wallis ) would be preferable when comparing los between different levels in each risk factor .
as the los of each infant cared for within a hospital may not be independentfrom that of another infant in the same hospital , even after adjusting for the fixed risk factors under study , we accounted for this hospital - level clustering by using generalized linear mixed models with hospital of care accounted for as a random effect .
we used a randomly selected 50% of the elbw cohort ( n=1,006 ) for model building , with the remaining 50% serving as a validation set .
further details of model selection are noted in the appendix and online table e2 . in our review of the literature
as described in the appendix , these seven models were tested for goodness of fit and other characteristics .
model 6 , a generalized linear model with negative binomial distribution and model 2 , a linear mixed model using the log scale were considered the two optimal los models .
the results of primary analyses and tests for biases ( described below ) were similar for both models , and model 2 formed the basis for the remainder of the study ( see appendix ) .
in model 2 we use a mixed model to account for clustering within nicus and we use the log transformation of los to reconcile the non - normal distribution that results from the very long los of some elbws with modeling assumptions . having selected the linear mixed model using log transformation as the preferred solution to estimating risk adjustedlos , the model was applied to the whole study cohort .
we started from an unconditional means model , then added infant - level risk factors .
final risk factor selection for the model was determined by backward selection ( alpha < 0.05 significance level ) .
the model diagnostics and validation of model assumptions were checked again in the final model . when residuals for the final model fitting were examined , there were two extreme outlier observations ( with significantly higher dffits and cook s distance statistics ) .
these two extreme data points were excluded from analysis in order to optimize model fit . as a linear mixed model does not generate an r statistic , we estimated r by using the same covariates in a model without random effects . based on infant - level estimates , hospital level results
risk adjusted los at each hospital was computed in the following fashion : 
 ( a ) for each infant , estimate an expected los(b ) obtaina calibration factor and apply to infant expected los .
the calibration factor was obtained by the overall ratio of ( sum of actual los / sum of expected los ) across the whole study cohort , as described in previous studies in order to center the estimate around 1.0.(c ) sum infants actual los and their expected los obtained in the previous step by hospital(d ) divide hospitals actual los sum by expected los sum to obtain o / e ratio(e ) multiply hospitals o / e ratio by overall average los ( a ) for each infant , estimate an expected los ( b ) obtaina calibration factor and apply to infant expected los .
the calibration factor was obtained by the overall ratio of ( sum of actual los / sum of expected los ) across the whole study cohort , as described in previous studies in order to center the estimate around 1.0 .
( c ) sum infants actual los and their expected los obtained in the previous step by hospital ( d ) divide hospitals actual los sum by expected los sum to obtain o / e ratio ( e ) multiply hospitals o / e ratio by overall average los in order to examine variation by hospital , we estimated risk - adjusted observed to expected los ratios for each hospital . in order to facilitate root cause analysis of a nicu s infants with excessive los , outliers for los at the individual patient level
these infants were considered individual outliers . as infants who died were excluded from the analysis , we wanted to examine the possibility that hospitals that may have shorter mean los could have higher rates of mortality , thereby having a less acutely ill surviving population of infants that would be biased toward shorter los . to examine this potential bias on average los
the mortality rate was computed from an established cpqcc algorithm , a modification of a vermont oxford network model , using all elbw infants excluding delivery room death in the cpqcc dataset between 20082010 ( n=7,270 ) . observed to expected mortality ratios were computedand their relationship to hospital los was evaluated using weighted correlation . in this regression model , each hospital was assigned a weight of the number of eligible patients cared for at that hospital .
another potential source of bias could be the exclusion of infants who are transferred out to lower levels of care .
when elbw infants cared for at level iii centers are transferred out for feeding and growing at lower levels of care , it is possible that the infants remaining in that nicu may present more complex cases who could be biased toward longer los .
similar to the analysis of mortality , we considered the relationship of transfer - out rates for feeding and growing with average los by hospital for nicus with weighted correlation .
all statistical analyses were conducted using sas version 9.3 ( sas , cary , nc ) .
we used data from the california perinatal quality care collaborative ( cpqcc ) , a voluntary collective of 129 nicus that report demographic and clinical data into a central repository .
more than 90% of nicus in california participated in this collaborative between january 1 , 2008 , and december 31 , 2010 .
member nicus collect data on their patients in a prospective manner , and each record undergoes a variety of range , logic , and missing data checks .
the period of analysis included infants cared for in cpqcc nicus for the three years , 2008 to 2010 .
inclusion criteria were infants with birth weight 4011,000 grams who were cared for at a cpqcc hospital from birth or within the first day after delivery , until discharge to home .
exclusion criteria were patients who died , those with congenital anomalies , and infants who had any surgery except circumcision .
the primary outcome ofinterest was length of stay ( los ) , defined as the number of days from admission to discharge . for infants
those infants not cared for at a cpqcc hospital on the first day , the first day of hospital stay at a cpqcc hospital counted as day one .
the following baseline characteristics were considered in risk adjustment : infant characteristics ( sex , birth weight , gestational age , small for gestational age ( sga ) ) , maternal and intrapartum variables ( maternal age , race / ethnicity , prenatal care , location of birth ( inborn / outborn ) , antenatal steroid use , multiple birth , mode of delivery , antenatal conditions ) , and delivery room variables ( 5 minute apgar scores ) . in considering sga (
defined as birth weight < 10th percentile for gestational age ) , it was noted that sga infants had shorter los , particularly those with gestational age > = 30 weeks . as birth weight
< 1,000 grams was an inclusion criterion , we used an approach that accounted for the birth weight restriction of the eligible cohort .
the sga variable was categorized into 2 different groups : 1 ) sga infants < 30 weeks of gestational age , and 2 ) sga infants > = 30 weeks ) .
we chose 30 weeks as the cut point because the 3rd percentile of 1,000 gram infants is approximately 30 weeks .
the rationale was to distinguish small size by infants developmental stage in terms of gestational age .
previous research has shown a survival advantage for sga neonates of < 31 weeks associated with cesarean delivery .
we examined collinearity among the model variables under consideration , as collinearity may influence estimation and inference for the primary parameters and lead to a loss of power , considering variance inflation factors and tolerance ( see appendix and table e1 for details on this selection process ) . from this analysis
, we included birth weight and sga , but not gestational age as explanatory variables in models .
skewness in the empirical distribution with a long positive side tail suggested that the use of the non - parametric test ( kruskal - wallis ) would be preferable when comparing los between different levels in each risk factor .
as the los of each infant cared for within a hospital may not be independentfrom that of another infant in the same hospital , even after adjusting for the fixed risk factors under study , we accounted for this hospital - level clustering by using generalized linear mixed models with hospital of care accounted for as a random effect .
we used a randomly selected 50% of the elbw cohort ( n=1,006 ) for model building , with the remaining 50% serving as a validation set .
further details of model selection are noted in the appendix and online table e2 . in our review of the literature
as described in the appendix , these seven models were tested for goodness of fit and other characteristics .
model 6 , a generalized linear model with negative binomial distribution and model 2 , a linear mixed model using the log scale were considered the two optimal los models .
the results of primary analyses and tests for biases ( described below ) were similar for both models , and model 2 formed the basis for the remainder of the study ( see appendix ) .
in model 2 we use a mixed model to account for clustering within nicus and we use the log transformation of los to reconcile the non - normal distribution that results from the very long los of some elbws with modeling assumptions .
having selected the linear mixed model using log transformation as the preferred solution to estimating risk adjustedlos , the model was applied to the whole study cohort .
we started from an unconditional means model , then added infant - level risk factors .
final risk factor selection for the model was determined by backward selection ( alpha < 0.05 significance level ) .
the model diagnostics and validation of model assumptions were checked again in the final model . when residuals for the final model fitting were examined , there were two extreme outlier observations ( with significantly higher dffits and cook s distance statistics ) .
these two extreme data points were excluded from analysis in order to optimize model fit . as a linear mixed model does not generate an r statistic , we estimated r by using the same covariates in a model without random effects . based on infant - level estimates ,
risk adjusted los at each hospital was computed in the following fashion : 
 ( a ) for each infant , estimate an expected los(b ) obtaina calibration factor and apply to infant expected los .
the calibration factor was obtained by the overall ratio of ( sum of actual los / sum of expected los ) across the whole study cohort , as described in previous studies in order to center the estimate around 1.0.(c ) sum infants actual los and their expected los obtained in the previous step by hospital(d ) divide hospitals actual los sum by expected los sum to obtain o / e ratio(e ) multiply hospitals o / e ratio by overall average los ( a ) for each infant , estimate an expected los ( b ) obtaina calibration factor and apply to infant expected los .
the calibration factor was obtained by the overall ratio of ( sum of actual los / sum of expected los ) across the whole study cohort , as described in previous studies in order to center the estimate around 1.0 .
( c ) sum infants actual los and their expected los obtained in the previous step by hospital ( d ) divide hospitals actual los sum by expected los sum to obtain o / e ratio ( e ) multiply hospitals o / e ratio by overall average los in order to examine variation by hospital , we estimated risk - adjusted observed to expected los ratios for each hospital . in order to facilitate root
cause analysis of a nicu s infants with excessive los , outliers for los at the individual patient level were identified based on studentized residuals outside of the 2 to + 2 range .
as infants who died were excluded from the analysis , we wanted to examine the possibility that hospitals that may have shorter mean los could have higher rates of mortality , thereby having a less acutely ill surviving population of infants that would be biased toward shorter los . to examine this potential bias on average los
the mortality rate was computed from an established cpqcc algorithm , a modification of a vermont oxford network model , using all elbw infants excluding delivery room death in the cpqcc dataset between 20082010 ( n=7,270 ) . observed to expected mortality ratios were computedand their relationship to hospital los was evaluated using weighted correlation . in this regression model , each hospital was assigned a weight of the number of eligible patients cared for at that hospital .
another potential source of bias could be the exclusion of infants who are transferred out to lower levels of care .
when elbw infants cared for at level iii centers are transferred out for feeding and growing at lower levels of care , it is possible that the infants remaining in that nicu may present more complex cases who could be biased toward longer los .
similar to the analysis of mortality , we considered the relationship of transfer - out rates for feeding and growing with average los by hospital for nicus with weighted correlation .
all statistical analyses were conducted using sas version 9.3 ( sas , cary , nc ) .
there were 2,012 elbw infants in the eligible cohort cared for in 111 hospitals ( figure e1  in online supplementary materials ) , with an average birth weight of 830 grams ( sd 126 ) and gestational age 27.2 weeks ( sd 2.0 ) .
the majority of patients were cared for in community level nicus ( 64% ) , with 32% of infants cared for in regional nicus , 1% in intermediate nicus , and 4% in non - classified nicus .
the median los was 79 days , with a range of 23 to 219 days ( table e3  in online supplementary materials ) .
infants who had antenatal steroid exposure and were born by cesarean delivery had shorter mean los . over the 3-year study period , los variation between years was not statistically significant ( p=0.11 ) .
the results of the risk adjusted los model are shown in table 1 , and the distribution of observed and predicted los is shown in figure 1 . birth weight ,
gender , sga , ethnicity , antenatal steroid use , fetal distress , maternal hypertension , and 5 minute apgar score were significant risk factors for longer los .
the random hospital intercept was strongly significant , indicating significant variation in los between hospitals .
the model without hospital included as a random effect had r of 0.45 . in order to assess the possibility of bias for nicus that had higher mortality rates
leading to shorter los , we investigated the relationship between risk - adjusted mortality to the mean risk - adjusted los in hospitals ( figure 2 )
. the mean mortality rate at the nicu level was 15.3% with interquartile range : 8.7% to 22.4% ( risk adjusted mortality rate mean 16.0% , interquartile range 9.9% to 23.5% ) .
the regression fit between los and risk adjusted mortality rate was poor with a slope of 0.00351and r = 0.0207 , indicating no relationship between mortality rate and length of stay by nicu . as our study included only infants who were discharged home , another potential bias that we considered was that hospitals that transferred more infants to lower levels of care for feeding and growing may have higher los .
the average transfer - out rate for growth and discharge planning was 7.5% overall for elbw infants , with range between 0 to 31% .
the mean adjusted los for nicus with transfer out rate < 10% ( n=96 nicus ) was 80.7 days ( interquartile range 79.1  83.0 ) while the mean adjusted los for nicus with transfer out rate greater than or equal to 10% ( n=15 nicus ) was 82.5 days ( interquartile range 80.4  83.5 ) .
figure 3 displays the relationship between transfer - out rate and risk - adjusted los .
overall , the impact of transfer - out rate on our risk - adjusted los appeared limited .
after risk adjustment , the mean nicu los among 27 nicus with more than 20 eligible patientswas 81.3 days with an interquartile range of 80.781.8 days and a total range of 78.283.7 days .
for the 84 centers with  20 eligible infants , the mean adjusted los was 81.8 days with a hospital interquartile range of 78.783.7 days and a total range of 54.0104.5 days .
if these smaller nicus were considered a single unit , the observed to expected ratio would be 1.003 . at the patient level , there were 91individual outliers identified of which 56 had observed los exceeding predicted .
using a large population dataset , we developed a risk adjustment model for nicu los for elbw infants , for estimating individual patient los , and for comparing nicu variation .
the methodology employed in this study also forms a basis for further development of los models for other nicu sub - populations .
there was wide variance in los by birth weight , by gestational age , and other factors ( table e3 ) .
these differences should be noted as various stakeholders may consider los to be a benchmark of quality of care , highlighting the need for the development of optimal risk adjustment models .
because of the non - normality of los , we tried several forms of transformation as well as different conditional distribution models to fit the data .
the log - scale model was chosen as the primary model due to favorable model characteristics .
we made certain exclusions in order to get a group of elbw infants that would best represent the typical los of a patient in this nicu subgroup .
as deaths can occur soon after birth and all the way up to several weeks or months of age , the  noise  in this group of patients would make assessment of nicu los difficult .
however , by excluding deaths , there was the possibility that nicus that perform poorly in survival could be perceived as having lower los , as their sickest patients would not be counted in the analysis . in a study by cotten
examining infants born at or before 28 weeks gestational age , there was noted to be an inverse relationship of los and mortality , with centers having less infants with prolonged los more likely to have higher mortality .
in contrast , across 111 nicus that included regional , community , and intermediate nicus , and caring for a few to many elbw patients , we found less concern for this potential bias , as there was no correlation between los and risk - adjusted mortality . in the study by cotten
, the cohort included births during 1998 to 2001 , and mortality rate was higher overall at 28.5% vs. 15.3% in our cohort .
although we have addressed one aspect of the question of how deaths are incorporated into analyses on los , this is a complex issue that will require ongoingstudy to develop optimal strategies for assessment , as well as study in other settings where the death to los relationship may differ .
on the other hand , nicus that transfer many of their less acute patients to nicus with lower level of care could falsely have an increased los , as their sickest patients would be over - represented in the analysis .
some centers may have a networkof several lower level nicus at outlying hospitals for transferring convalescing infants , whereas other nicus may not have that capacity . within our heterogeneous group of nicus
, we did not find a significant relationship between risk - adjusted los and transfer - out rates for feeding and growing , alleviating this concern .
however , as our analysis only concerned elbw infants , these potential biases would need to be further tested for other nicu populations .
further work in evaluating models that include transferred patients , both transferred in to a higher level of care , as well as those transferred out to lower level of care may increase the potential to assess larger systems of care .
there may be several limitations to our analysis in regard to the data available for modeling
. there may be some socio - demographic factors that were not accounted for in the dataset , such as family income .
although we accounted in part for severity of illness by including variables such as birth weight and apgar score in the model , other measures obtained at time points such as admission to the nicu or in the early hospital course could potentially strengthen the model fit . on the other hand , as antenatal and delivery room management could impact both severity of illness and length of stay , the ultimate purpose of the los assessment may guide inclusion of such variables in modeling . in our random effects model ,
the hospital intercept was highly significant , indicating that there are opportunities for reducing the variation of elbw los across nicus .
when the hospital effect was removed from the model , the explanatory power of the model ( r = 0.45 ) was comparable to that reported in a previous study of nicu all patient diagnosis - related groups , a classification system for inpatient hospital care .
as our model incorporated both clinically and statistically significant maternal and neonatal factors for risk adjustment , it may be useful as a basis for assessing and comparing nicu los for an individual center across time , as well as between centers .
we have developed a patient - level risk - adjusted model for assessing los for elbw infants , which additionally accounts for clustering of outcomes by nicu ( hospital ) .
our study also allows hospitals to assess their risk adjusted los and will give the ability for each nicu to identify infants whose los are extreme and could serve as a starting point for root cause analysis of excessive los
. this work forms the basis for developing models for los for other nicu sub - populations .
further research will focus on optimal methods to compare hospitals in los for the purpose of quality improvement , public reporting , and optimizing payment structures .
this work also forms the foundation for studying the relationship of los to short- and long - term clinical outcomes , and future research on how family resources and competency , nicu processes , and hospital factors such as level of care may influence risk - adjusted los .
* discharged home with eligibility criteria ( inborn or outborn admitted within 24 hours from birth ) * * transfer or discharge home cases where eligibility criteria were not met
 objectiveto develop a length of stay ( los ) model for extremely low birth weight ( elbw ) infants.study designwe included infants from the california perinatal quality care collaborative with birth weight 4011,000 grams who were discharged to home . 
 exclusion criteria were congenital anomalies , surgery , and death . 
 los was defined as days from admission to discharge . as patients who died or were transferred to lower level of care 
 were excluded , we assessed correlation of hospital mortality rates and transfers torisk adjusted los.resultsthere were 2,012 infants with median los 79 days ( range 23219 ) . 
 lower birth weight , lack of antenatal steroids , and lower apgar score were associated with longer los . 
 there was negligiblecorrelation between risk - adjusted los and hospital mortality rates ( r = 0.0207 ) and transfer - out rates ( r = 0.121).conclusionparticularly because elbw infants have extended hospital stays , identification of unbiased and informative risk - adjusted los for these infants is an important step in benchmarking best practice and improving efficiency in care . 
Introduction
Methods
Data source, study population, and patient characteristics
Selection of explanatory variables
Model selection
Final model construction
Tests for biases
Results
Discussion
Conclusion
Supplementary Material