the mortality rate is one of the key indicators of an intensive care unit ( icu ) quality .
however , the severity of patients ' illnesses , comorbidities , and demographics all strongly affect the mortality rate ( 1 ) .
thus a comparison of unadjusted mortality rate among icus without consideration of severity or other case - mix variables might provide an incorrect estimate of icu quality . in a multicenter study ,
the crude mortality rate of tertiary pediatric icus was four times higher than that of non - tertiary pediatric icus ( 2 ) . the latter does not suggest that the performance of the tertiary pediatric icu was poor , as tertiary hospitals are usually referral hospitals to which more severe patients are referred . therefore ,
when we compare mortality data of certain units or patients ' groups for the purpose of quality care or research , the mortality data must be adjusted .
mortality data on icu patients is usually adjusted using mortality prediction models . in general pediatric icus , the pediatric index of mortality ( pim ) 3 , pediatric risk of mortality ( prism ) iii ,
are commonly used mortality prediction models ( 345 ) , whereas the acute physiology and chronic health evaluation ( apache ) , simplified acute physiology score and mortality probability model are commonly used in adult icus ( 6 ) .
regional variations , such as different national health systems , different populations , or disease prevalence , might affect the performance of these scoring systems .
thus , some validation studies of the applicability of these scoring systems to korean adult patients have been conducted in korea ( 789 ) . to the best of our knowledge , there has been no validation study of pim3 scoring system for pediatric patients in korea .
another consideration when applying pim score in korean icus is the age of the patients .
the developmental data of pim score were validated for children aged < 16 years at the time of icu admission , but patients aged < 18 years are admitted to pediatric icus in korea .
therefore , the aims of this study were : 1 ) to validate the recently updated pim3 score for korean pediatric patients ; and 2 ) to validate the pim3 score for patients in an extended age group ( < 18 years ) .
this was a retrospective cohort study of patients admitted to the pediatric intensive care unit ( picu ) of a single tertiary institute , from 1st january 2010 to 31st december 2014 .
immediate postoperative cardiac patients are usually admitted to another dedicated pediatric cardiac surgical unit , but can be admitted to this picu when medical support is more emphasized than postoperative care .
many surgical patients , especially in a post organ transplantation state , are admitted to the specialty icu shared by adult patients .
the inclusion and exclusion criteria were similar to those of previous studies of the pim and revised versions ( pim2 and pim3 ) ( 41011 ) , except for the age of the patients .
in contrast to the criteria in previous studies , which included only patients aged < 16 years , the present study included patients aged < 18 years .
readmissions were considered as new admissions , but patients transferred to other hospital units were excluded .
the data were collected by a licensed medical records officer and reviewed by researchers ( intensive care physicians ) collaborating in the study .
data on the following parameters were collected to calculate the pim3 score : 1 ) systolic blood pressure ( sbp ) , mmhg ( unknown = 120 ) ; 2 ) pupillary reactions to bright light ( > 3 mm and both fixed = 1 ; other or unknown = 0 ) ; 3 ) ( [ fio2  100]/pao2 ) .
pao2 mmhg , fio2 at the time of pao2 if oxygen via endotracheal tube or headbox ( fio2 or pao2 unknown , [ { fio2  100}/pao2 ] = 0.23 ) ; 4 ) base excess in arterial or capillary blood , mmol / l ( unknown = 0 ) ; 5 ) mechanical ventilation at any time during the first hour in icu ( no = 0 ; yes = 1 ) ; 6 ) elective admission to icu ( no = 0 ; yes = 1 ) ; 7 ) recovery from surgery or a procedure is the main reason for icu admission ( no = 0 ; yes , recovery from a bypass cardiac procedure = 1 ; yes , recovery from a non - bypass cardiac procedure = 2 ; yes , recovery from a noncardiac procedure = 3 ) ; 8) low - risk diagnosis ( no or in doubt = 0 ; yes = 1 ) ; 9 ) high - risk diagnosis ( no or in doubt = 0 ; yes = 1 ) ; and 10 ) very high - risk diagnosis ( no or in doubt = 0 ; yes = 1 ) .
low - risk diagnosis consists of asthma , bronchiolitis , croup , obstructive sleep apnea , diabetic ketoacidosis , and seizure disorder .
high - risk diagnosis consists of spontaneous cerebral hemorrhage , cardiomyopathy or myocarditis , hypoplastic left heart syndrome , neurodegenerative disorder , and necrotizing enterocolitis .
very high - risk diagnosis consists of cardiac arrest preceding icu admission , severe combined immune deficiency , leukemia or lymphoma after first induction , bone marrow transplant recipient , and liver failure .
patients with multiple diagnoses were assigned to only 1 risk - diagnosis group , which has the highest risk score .
pim3 score and probability of death were calculated by formulas presented in the development study ( 4 ) . pim3 score = ( 3.8233  pupillary reaction ) + ( 0.5378  elective admission ) + ( 0.9763  mechanical ventilation ) + ( 0.0671  [ absolute { base excess } ] ) + ( 0.0431  sbp ) + ( 0.1716  [ sbp/1,000 ] ) + ( 0.4214  [ { fio2  100}/pao2 ] )  ( 1.2246  bypass cardiac procedure )  ( 0.8762  non - bypass cardiac procedure )  ( 1.5164  noncardiac procedure ) + ( 1.6225  very high - risk diagnosis ) + ( 1.0725  high - risk diagnosis )  ( 2.1766  low - risk diagnosis )  1.7928 . probability of death = exp ( pim3 score)/[1 + exp ( pim3 score ) ] .
data on parameters that differed in the pim2 version , including the main reason for icu admission , cardiac bypass surgery , diagnosis category ( low- , high - risk ) , were collected as described in the developmental study of pim2 ( 11 ) .
physiological variables from the time of first icu contact to 1 hour after arrival in the icu were recorded .
the formulas and coefficients presented in previous pim2 and pim3 studies were used to calculate the predicted mortality of each patient ( 411 ) . the sas program ( ver .
9.3 ; sas institute , cary , nc , usa ) was used for all the statistical analyses .
all tests were 2-tailed , and a value of p < 0.05 was considered statistically significant .
descriptive statistics were used to present age , gender , length of stay , observed mortality , and predicted mortality . to evaluate the performance of the pim2 and pim3 scoring systems in patients aged < 18 years , we assessed the calibration and discrimination power of the score .
we also assessed the performance in patients aged < 16 years and in diagnostic subgroups of cardiac , hemato - oncologic , and respiratory diagnosis , as well as the patients under the other diagnosis .
the hosmer - lemeshow goodness - of - fit ( gof ) test was used to evaluate the calibration of the scoring system .
calibration refers to the level of agreement between individual probabilities and actual outcomes ( 12 ) .
the gof was assessed by dividing the patients into 10 groups according to their predicted mortality risk ( c statistic ) , with a similar number of patients in each group .
the expected number of deaths as predicted by pim3 was compared with the observed number of deaths in each group .
the discriminatory power ( i.e. , the ability to distinguish between survival and death ) was assessed by calculating the area under the receiver - operating characteristic ( roc ) curve ( c - index ) .
a c - index of > 0.70 was regarded as acceptable , and a value of > 0.8 was regarded as good discrimination ( 13 ) .
the present study protocol was reviewed and approved by the institutional review board of samsung medical center ( irb no .
this was a retrospective cohort study of patients admitted to the pediatric intensive care unit ( picu ) of a single tertiary institute , from 1st january 2010 to 31st december 2014 .
immediate postoperative cardiac patients are usually admitted to another dedicated pediatric cardiac surgical unit , but can be admitted to this picu when medical support is more emphasized than postoperative care .
many surgical patients , especially in a post organ transplantation state , are admitted to the specialty icu shared by adult patients .
the inclusion and exclusion criteria were similar to those of previous studies of the pim and revised versions ( pim2 and pim3 ) ( 41011 ) , except for the age of the patients .
in contrast to the criteria in previous studies , which included only patients aged < 16 years , the present study included patients aged < 18 years .
readmissions were considered as new admissions , but patients transferred to other hospital units were excluded .
the data were collected by a licensed medical records officer and reviewed by researchers ( intensive care physicians ) collaborating in the study .
data on the following parameters were collected to calculate the pim3 score : 1 ) systolic blood pressure ( sbp ) , mmhg ( unknown = 120 ) ; 2 ) pupillary reactions to bright light ( > 3 mm and both fixed = 1 ; other or unknown = 0 ) ; 3 ) ( [ fio2  100]/pao2 ) .
pao2 mmhg , fio2 at the time of pao2 if oxygen via endotracheal tube or headbox ( fio2 or pao2 unknown , [ { fio2  100}/pao2 ] = 0.23 ) ; 4 ) base excess in arterial or capillary blood , mmol / l ( unknown = 0 ) ; 5 ) mechanical ventilation at any time during the first hour in icu ( no = 0 ; yes = 1 ) ;
6 ) elective admission to icu ( no = 0 ; yes = 1 ) ; 7 ) recovery from surgery or a procedure is the main reason for icu admission ( no = 0 ; yes , recovery from a bypass cardiac procedure = 1 ; yes , recovery from a non - bypass cardiac procedure = 2 ; yes , recovery from a noncardiac procedure = 3 ) ; 8) low - risk diagnosis ( no or in doubt = 0 ; yes = 1 ) ; 9 ) high - risk diagnosis ( no or in doubt = 0 ; yes = 1 ) ; and 10 ) very high - risk diagnosis ( no or in doubt = 0 ; yes = 1 ) .
low - risk diagnosis consists of asthma , bronchiolitis , croup , obstructive sleep apnea , diabetic ketoacidosis , and seizure disorder .
high - risk diagnosis consists of spontaneous cerebral hemorrhage , cardiomyopathy or myocarditis , hypoplastic left heart syndrome , neurodegenerative disorder , and necrotizing enterocolitis .
very high - risk diagnosis consists of cardiac arrest preceding icu admission , severe combined immune deficiency , leukemia or lymphoma after first induction , bone marrow transplant recipient , and liver failure .
patients with multiple diagnoses were assigned to only 1 risk - diagnosis group , which has the highest risk score .
pim3 score and probability of death were calculated by formulas presented in the development study ( 4 ) .
pim3 score = ( 3.8233  pupillary reaction ) + ( 0.5378  elective admission ) + ( 0.9763  mechanical ventilation ) + ( 0.0671  [ absolute { base excess } ] ) + ( 0.0431  sbp ) + ( 0.1716  [ sbp/1,000 ] ) + ( 0.4214  [ { fio2  100}/pao2 ] )  ( 1.2246  bypass cardiac procedure )  ( 0.8762  non - bypass cardiac procedure )  ( 1.5164  noncardiac procedure ) + ( 1.6225  very high - risk diagnosis ) + ( 1.0725  high - risk diagnosis )  ( 2.1766  low - risk diagnosis )  1.7928 . probability of death = exp ( pim3 score)/[1 + exp ( pim3 score ) ] . data on parameters that differed in the pim2 version , including the main reason for icu admission , cardiac bypass surgery , diagnosis category ( low- , high - risk ) , were collected as described in the developmental study of pim2 ( 11 ) .
physiological variables from the time of first icu contact to 1 hour after arrival in the icu were recorded .
the formulas and coefficients presented in previous pim2 and pim3 studies were used to calculate the predicted mortality of each patient ( 411 ) .
9.3 ; sas institute , cary , nc , usa ) was used for all the statistical analyses .
all tests were 2-tailed , and a value of p < 0.05 was considered statistically significant .
descriptive statistics were used to present age , gender , length of stay , observed mortality , and predicted mortality . to evaluate the performance of the pim2 and pim3 scoring systems in patients aged < 18 years , we assessed the calibration and discrimination power of the score .
we also assessed the performance in patients aged < 16 years and in diagnostic subgroups of cardiac , hemato - oncologic , and respiratory diagnosis , as well as the patients under the other diagnosis .
the hosmer - lemeshow goodness - of - fit ( gof ) test was used to evaluate the calibration of the scoring system .
calibration refers to the level of agreement between individual probabilities and actual outcomes ( 12 ) .
the gof was assessed by dividing the patients into 10 groups according to their predicted mortality risk ( c statistic ) , with a similar number of patients in each group .
the expected number of deaths as predicted by pim3 was compared with the observed number of deaths in each group .
the discriminatory power ( i.e. , the ability to distinguish between survival and death ) was assessed by calculating the area under the receiver - operating characteristic ( roc ) curve ( c - index ) .
a c - index of > 0.70 was regarded as acceptable , and a value of > 0.8 was regarded as good discrimination ( 13 ) .
the present study protocol was reviewed and approved by the institutional review board of samsung medical center ( irb no .
of those , 93 patients were excluded because they were 18 years or older at the time of icu admission , and one patient was excluded because of an incomplete medical record . the final study consisted of 1,710 patients , with approximately 340 patients included annually in the study . in this study group , 208 cases ( 12.1% )
were readmitted to the picu during the hospital stay ( median interval of readmission : 9 days ) . as for the age ,
1,656 patients were younger than 16 years of age , and the median age was 1.58 years ( interquartile range [ iqr ] : 0.337.85 years ) .
the most common diagnostic category was cardiovascular disease , followed by hemato - oncological disease and respiratory disease .
the other diagnostic category included hepatic dysfunction or gastrointestinal disease ( 57 case , 3.3% ) , immunodeficiency disease ( 52 case , 3.0% ) , renal disease ( 44 case , 2.6% ) , metabolic disease ( 39 case , 2.3% ) , and unclassified disease ( 12 case , 0.7% ) .
the observed hospital mortality rate was 8.47% , which was higher than the predicted mortalities calculated by both the pim2 and pim3 formulas ( table 1 ) .
iqr = interquartile range , icu = intensive care unit , iqr = interquartile range , pim = pediatric index of mortality .
the unadjusted mortality rates and predicted mortality rates of both age groups ( < 18 years and < 16 years ) were similar ( table 2 )
. the discriminatory power of the pim3 model was acceptable ( c - index > 0.70 ) and better than that of the pim2 model ( c - index = 0.70 ) .
1 , the calibration of both the pim2 and pim3 models was good for both age groups ( p > 0.05 ) , but the  value for the pim2 model was higher than the value for the pim3 model .
in addition , the expected mortality rates were closer to the observed mortality rates in pim2 model ( table 2 ) .
when we divided the patients into 10 similar sized groups according to their predictive mortality risk , both the average predicted mortality and observed mortality of approximately 90% of the patients was lower than 20% ( fig .
1 ) . there were some differences in the predicted and observed mortality in the subgroups , but these were acceptable according to the hosmer - lemeshow gof test .
gof = goodness - of - fit , roc = receiver - operating characteristic , pim = pediatric index of mortality , ci = confidence interval .
 expected  hospital mortality ( % ) calculated by the pim3 model and  observed  hospital mortality ( % ) , together with the corresponding patient number , are presented .
the patients were divided into 10 similar sized groups based on predicted mortality . in the diagnostic subgroups ,
the mortality rate of the hemato - oncological patients was 2 to 5 times higher ( 18.73% ) compared to that of the other subgroups ( table 3 ) .
the calibration was acceptable for all the subgroups , including the cardiac , respiratory , and hemato - oncological groups .
however , as shown in table 3 , the discriminatory power of the c - index for the hemato - oncological patients was lower than acceptable ( 0.666 ) . gof = goodness - of - fit , roc = receiver - operating characteristic , pim = pediatric index of mortality , ci = confidence interval .
this is the first study to validate the pim3 scoring system in a pediatric icu in korea .
we validated the pim3 score in a korean pediatric icu population using the formulas and coefficients presented in the developmental study of pim3 score ( 4 ) .
the present study showed that the pim3 had acceptable discrimination ability ( c - index = 0.76 ) and good calibration , with a  value of 9.4 in those aged < 18 years ( p = 0.313 ) .
several studies have performed regional validations of the pim2 score in many countries ( 14151617181920 ) . in japan ,
a single - center validation study reported that the pim2 had an excellent discrimination power ( c - index = 0.92 ) and good calibration with a hosmer - lemeshow gof test  value of 4.8 ( p = 0.440 ) ( 19 ) .
only 1 recent study has performed a regional validation of the pim3 score , which is the newest version of the pim score , in italy ( 21 ) .
however , it includes a more complicated risk - weighting diagnosis , with the diagnosis classified into 3 risk groups ( low , high , and very high ) rather than two ( low and high ) , as in the previous score . in this study ,
the discriminatory power of the pim3 score was acceptable ( c - index of > 0.7 ) .
it was similar to the value in an italian validation study , which reported that pim3 more accurately predicted the mortality risk than pim2 ( 21 ) .
therefore , the pim3 score may be a reasonable choice for pediatric icus in korea , despite its additional complexity .
in previous validation studies , the area under the roc curve ( c - index ) was acceptable , but the values varied between 0.79 and 0.92 .
the c - indices in those studies were higher than the value reported in this study , which means the indices had better discriminatory power .
the variation may be explained regional differences in study populations . in the development study of pim2 and pim3 ,
cardiac , respiratory , and non - cardiac postoperative groups were most common diagnostic groups ( 411 ) .
in contrast , in the present study , cardiac , hemato - oncological , and respiratory groups were the most prevalent diagnostic groups .
the hemato - oncological subgroup was the second most common subgroup in the present study , and it had the lowest c - index ( 0.66 ) compared to the other subgroups ( 0.740.83 ) .
the poor performance of pim and prism score in predicting mortality of oncological patients was reported in several studies ( 22232425 ) .
one study reported that both prism and pim score underestimated the mortality of oncological patients ( 24 ) .
another study suggested poor performance of prism3 in predicting mortality of pediatric stem cell transplantation patients ( 25 ) . in a study of adults , the apache ii scoring system also underestimated the mortality of bone marrow transplant recipients when the apache ii score was low ( 26 ) .
the underestimation of mortality rate might be caused by the innate limitation of the general mortality prediction score , which has one - size - fits - all property .
for example , the predicted mortality of a respiratory failure patient , which is measured by pao2 , fio2 and the usage of mechanical ventilation ( in pim3 score ) , pao2 , and pco2 ( in prism3 score ) , can be calculated in the same way for a previously healthy child with viral pneumonia and an oncological patient with cytomegalovirus / fungal pneumonia , pulmonary veno - occlusive disease , or pulmonary graft - versus - host disease at the time of admission .
since the latter patient usually has very poor prognosis ( 24 ) , the observed mortality rate might be very different between the 2 patients .
a general mortality prediction model might be inappropriate for use with patients in specialized oncological icus .
instead , a disease - specific scoring system , reflecting specific prognosis , might be more useful ( 27 ) .
the observed mortality was 8.47% , and it was higher than the reported observed mortality . high unadjusted mortality may be explained by the severity of illness of patients in the icu or the poor performance of the icu team . to clarify the actual cause ,
the standard mortality ratio ( smr ) , which is the observed mortality rate divided by the average predicted mortality , can be calculated ( 1 ) .
the smr is the most commonly used parameter of icu quality in western europe and is mandated by some countries ( 28 ) .
it can also be used to compare mortality data , follow changes over time , and evaluate the effect of interventions or events ( 2930731 ) . to calculate the smr , the predicted mortality must first be determined .
one advantage of the pim3 is that the formula and coefficients of predicted mortality is presented in the article and freely available ( 4 ) , compared to the mortality prediction formula of prism3 is commercially patented and only incomplete prism score is documented in the study report . in this study , the observed mortality was higher than the predicted mortality .
however , the smr was higher than 1 , suggesting that the performance of the icu was low .
the time patterns of admission ( i.e. , early admission before full disease progression or late admission when the disease is severe ) can affect the physiological parameters recorded at the time of icu admission and eventually the smr .
variation in the predicted mortality according to the scoring system can also influence the smr ( 32 ) . in latin america ,
some countries have tried to develop a calibrated mortality prediction model because of regional differences among icus ( 734 ) . despite its defects
for example , serial measurements of icu smrs can be used to monitor internal quality improvements in hospitals . at the national level
, the smr of pediatric icus can be measured every 510 years to evaluate the improvement of the quality of korean pediatric icus .
first , this was a single center study conducted at a tertiary hospital . thus the findings may not be generalizable to the entire korean pediatric population . however , there are only a few pediatric icus in korea , all of which are located in tertiary hospitals , so our study population might be similar to other pediatric icu populations in korea .
another problem is that many pediatric patients in korea are admitted to adult icus , but patients aged < 18 years are usually excluded in research of adult patients . to evaluate all the pediatric patients in icus , including pediatric patients in adult icus , multicenter studies , including primary and secondary hospitals ,
due to the diversity of the patients , a one - size - fits - all scoring system was used in the evaluation of the icu . in reality , this type of system is not suited to all patients , especially small specific subgroups .
the third limitation is the retrospective data collection , although every effort was made to validate the data thoroughly . in conclusion , the performance of the pim3 scoring system in korean patients aged < 18 years was good
. however , the pim3 may not be an appropriate quality measure for subgroups with hemato - oncological disease or oncology - specific icus because of its low discriminatory power . a multicenter study that includes pediatric patients in both adult and pediatric icus
 to compare mortality rate , the adjustment of case - mix variables is needed . 
 the pediatric index of mortality ( pim ) 3 score is a widely used case - mix adjustment system of a pediatric intensive care unit ( icu ) , but there has been no validation study of it in korea . 
 we aim to validate the pim3 in a korean pediatric icu , and extend the validation of the score from those aged 016 to 018 years , as patients aged 1618 years are admitted to pediatric icu in korea . 
 a retrospective cohort study of 1,710 patients was conducted in a tertiary pediatric icu . to validate the score , 
 the discriminatory power was assessed by calculating the area under the receiver - operating characteristic ( roc ) curve , and calibration was evaluated by the hosmer - lemeshow goodness - of - fit ( gof ) test . 
 the observed mortality rate was 8.47% , and the predicted mortality rate was 6.57% . 
 for patients aged < 18 years , the discrimination was acceptable ( c - index = 0.76 ) and the calibration was good , with a 2 of 9.4 in the gof test ( p = 0.313 ) . the observed mortality rate in the hemato - oncological subgroup was high ( 18.73% ) , as compared to the predicted mortality rate ( 7.13% ) , and the discrimination was unacceptable ( c - index = 0.66 ) . in conclusion , 
 the pim3 performed well in a korean pediatric icu . 
 however , the application of the pim3 to a hemato - oncological subgroup needs to be cautioned . 
 further studies on the performance of pim3 in pediatric patients in adult icus and pediatric icus of primary and secondary hospitals are needed . 
INTRODUCTION
MATERIALS AND METHODS
Study design
Data collection
Statistical analysis
Ethics statement
RESULTS
DISCUSSION