Introduction
============

Pressure ulcers is a significant quality index of patient care and one of the most crucial challenges encountered in current health and medical treatment ([@B1]). Because incidence rate and prevalence rate of pressure ulcers is dependent on patients' characteristics, each study does not show same results, however, Prevalence rate of Pressure ulcers in Americans increased significantly, from7% in 2000 ([@B2]) to 15% in 2009 ([@B3]). In Canada, 25% of patients in acute medical institutions and 30% in non-acute medical institutions had bed sore outbreaks ([@B4]). Lastly, 11% of patients ([@B5]) and 28.2% of nervous critical patients in Korea ([@B6]) had pressure ulcers. Millions of Pressure ulcers outbreaks occur annually worldwide and 60million patients die of complication caused by Pressure ulcers ([@B7]).

Pressure ulcers decrease quality of life, while increasing and medical expenses sharply as the pressure ulcers takes a serious turn ([@B8]). In addition, the duration of hospital stay will be extended; patient's function recovery will be delayed ([@B9]), which may cause problems such as the need of additional nurse workforce ([@B10]). Therefore, active prevention is important in reducing the occurrence of pressure ulcers ([@B11]). Ideal pressure ulcers prevention is possible, when we accurately select a few high-risk pressure ulcers occurrences among the total patient group by using qualified pressure ulcer risk scales. It also can be solved when we conduct education for patient/guardian and nursing management with concentrated prevention activity, thereby maximizing the efficiency ([@B12]).

Norton scale was developed and applied in the clinic for the first time. This scale was found to predict pressure ulcers of senile patients in England ([@B13]). In 1973, Gosnell revised the Norton scale for inpatients in long-term care facilities ([@B14]). In 1984, Shannon developed a scale made up of eight items ([@B15]), and in 1985, England, the Waterlow scale was developed in England for senile patients in acute medical institutions ([@B16]). In 1987, the Braden scale was developed for senile patients in long-term care facilities in the USA, lastly ([@B17]); also, in 1991, the Cubbin& Jackson scale was developed for ICU Patients ([@B18]). Other countries developed pressure ulcer risk scales that are suitable for their own patients ([@B19]--[@B21]). Beside from the Braden scale, predictive validity verification of tool application progress or repetitive studies was not performed so that development of a new scale is required ([@B22]--[@B23]).

In this study, we compared and analyzed the Cubbin & Jackson, modified Braden, Norton, and Waterlow pressure ulcer risk scales. We also aimed to provide scientific evidence of the accuracy of predictions of pressure sores through a diagnostic method oriented meta-analysis for the use of various pressure ulcer assessment tools in the clinic.

Methods
=======

This study was conducted using the *Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy* from the Cochrane Collaboration ([@B24]) and *Preferred reporting items for systematic reviews and meta-Analyses* from Preferred Reporting Items for Systematic Reviews and Meta-Analyses \[PRISMA\] ([@B25]).

Strategy for considering studies for this search
------------------------------------------------

Criteria for considering studies for this review By using the pressure ulcer risk scale: Cubbin & Jackson, modified Braden, Norton and Waterlow, we included studies that evaluated predictive validity of pressure ulcers outbreak in the proper patient group. Risk of pressure ulcers was judged upon each criterion, Based on a cut-off point suggested by each researcher

Pressure ulcer outbreak was verified with a rating scale of objective skin condition by professional groups and individual scholars: USA, National Pressure Ulcer Advisory Panel \[NPUAP\], European Pressure Ulcer Advisory Panel \[EPUAP\], Agency for Health Care Policy and Research \[AHCPR\], Torrence Developmental Classification of Pressure Sore \[TDCPS\].

Search methods for identification of studies
--------------------------------------------

Electronic database search was conducted on October 1^st^, 2013. We used electronic databases, including KoreaMed, NDSL, KERIS, Ovid-Medline, Embase, Cochrane Library, and CINAHL Plus with Full Text. We added our national medical/nursing academic journal search. Search words used were 'pressure ulcer', 'decubitus ulcer', 'skin ulcer', 'bed sore', and 'risk assessment'.

Risk of bias in included studies and data extraction
----------------------------------------------------

The quality of literature was evaluated independently by two authors using QUADAS-II (Quality Assessment of Diagnostic Accuracy Studies-II) ([@B26]). On data extraction, we filled out: type of study, location of study, number of research institutes, gender distribution of subjects, age, duration of hospital stay, and cut-off point of each scale, from selected literature after deciding on the basic form of evidence table; in addition, diagnosis result of pressure ulcer risk scale value of TP, FP, FN, TN was written. On the basis of it, we wrote a 2×2 table of specification and recalculated sensitivity, specificity, positive LR, negative LR, DOR, and 95% confidence interval \[95% CI\] using '2-way Contingency Table Analysis' program.

Statistical analysis
--------------------

Studies were analyzed using meta-analysis with MetaDiSc 1.4. The general rule of the statistical model is to analyze with random effect estimation in order to reflect the differences between studies ([@B24]), and analyzed with total sensitivity, specificity, positive LR, negative LR, DOR, and sROC curve. In statistics of sROC curve, we recorded the examination accuracy through the Area under the Curve \[AUC\] and index Q^\*^value. From AUC figure, AUC=0.5 is noninformative examination; 0.5 \<AUC ≤0.7 is less accurate; 0.7 \<AUC ≤0.9 is moderate; 0.9 \<AUC \<1 is perfect examination ([@B27]), index Q^\*^reflecting sensitivity and specificity on ROC curve is judged on the basis of '1' when it corrects 100% ([@B28]). Heterogeneity between studies was verified with sight while having a common part in the confidence interval and estimation of efficacy through picture of a forest. In addition, we investigated it using I^2^ test, decreasing significance level to 5%; the standard of judgment of I^2^: I^2^ ≤25% has low heterogeneity; 25% \<I^2^ ≤75% has moderate heterogeneity; 25% \<I^2^ ≤75% has high heterogeneity ([@B29]).

Results
=======

Finally, 17 documents were selected. The detailed selection method is shown in [Figure 1](#F1){ref-type="fig"}. Seven documents ([@B1], [@B23], [@B30]--[@B34]) were about the Norton tool, 6 documents ([@B30]--[@B31], [@B34]--[@B37]) were about Waterlow, 6 documents ([@B23], [@B32], [@B38]--[@B41]) were about modified Braden, and the other 4 documents ([@B6], [@B40], [@B42]--[@B43]) were about Cubbin & Jackson ([Table 1](#T1){ref-type="table"}).

![Flow diagram of article selection](IJPH-45-122-g001){#F1}

###### 

Comparison of characteristics of pressure ulcer risk assessment tools

  -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
                                         **Norton scale**                                        **Waterlow scale**                               **Modified Braden scale**                                    **Cubbin & Jackson scale**
  -------------------------------------- ------------------------------------------------------- ------------------------------------------------ ------------------------------------------------------------ ------------------------------------------------------------
  Created year                           1962 [^1)^](#TFN2){ref-type="table-fn"}                 1985 [^2)^](#TFN3){ref-type="table-fn"}          1987 (by Braden et al.) [^3)^](#TFN4){ref-type="table-fn"}   1991 [^6)^](#TFN7){ref-type="table-fn"}

                                                                                                                                                  1991 (by Choi & Song) [^4)^](#TFN5){ref-type="table-fn"}     

                                                                                                                                                  1998 (by Pang & Wong) [^5)^](#TFN6){ref-type="table-fn"}     

  Specialty                              Older people/generic                                    Orthopaedic/generic                              Generic                                                      Intensive care

  Risk factors to assess pressure sore   Physical condition\                                     Sex\                                             Sensory perception\                                          Age\
                                         Mental status\                                          Age\                                             Moisture [^\*^](#TFN8){ref-type="table-fn"}\                 Weight\
                                         Activity Mobility [^\*^](#TFN8){ref-type="table-fn"}\   Build\                                           Activity Mobility [^\*^](#TFN8){ref-type="table-fn"}\        Skin condition\
                                         Continence [^\*^](#TFN8){ref-type="table-fn"}           Appetite\                                        Nutrition status\                                            Mental status Mobility [^\*^](#TFN8){ref-type="table-fn"}\
                                                                                                 Nurses' visual assessment of skin condition\     Friction/shear by Choi & Song\                               Nutrition\
                                                                                                 Mobility [^\*^](#TFN8){ref-type="table-fn"}\     Temperature\                                                 Respiration\
                                                                                                 Continence [^\*^](#TFN8){ref-type="table-fn"}\   Use of medication by Pang & Wong\                            Continence [^\*^](#TFN8){ref-type="table-fn"}\
                                                                                                 Factors contributing to tissue malnutrition\     Skin type Body build\                                        Hygiene\
                                                                                                 Neurologic deficits\                                                                                          Hemodynamic status
                                                                                                 Major surgery or trauma Medication                                                                            
  -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Notes.

Norton, McLaren & Exton-Smith (1962);

Waterlow (1985);

Bergstrom, Braden, Laguzza, & Holman (1987);

Choi & Song (1991)

Pang & Wong (1998);

Cubbin & Jackson (1991);

Common risk factors to assess pressure sore

Risk of bias in included studies
--------------------------------

None of the documents was estimated to have "high" risk of inequality after evaluating the quality of the 17 selected documents.

The Test of Validity and the Quality of Selected Documents of Pressure Ulcer risk scale Norton Scale
----------------------------------------------------------------------------------------------------

There were seven literature studies ([@B1], [@B23], [@B30]--[@B34]) reporting diagnostic accuracy of the Norton scale, involving 2,899 patients. All literature studies are prospective studies, except 1 ([@B33]) out of 7, involving inpatients who have no pressure ulcers. Regarding average age, there was one study about 50s ([@B32]), one study about 80s ([@B1]), three literature studies on 60s, and the two remaining literature studies ([@B30], [@B34]) that do not mention age. Based on standard reference, three literature studies ([@B23], [@B30]--[@B21]) use AHCPR, two of them ([@B32]--[@B33]) use NPUAP, EPUAP ([@B1]) and TDCPS ([@B34]) are used in one literature study separately. The three studies use cutoff point: 14points ([@B1], [@B32], [@B24]) and 16points ([@B23], [@B31], [@B34]), and one ([@B30]) utilizes 15points. Total sensitivity was 0.75 (95% CI 0.70, 0.79) ([Figure 2](#F2){ref-type="fig"}-E), and total specificity was 0.57 (95% CI 0.55, 0.59). The heterogeneity between studies was high, as 88.9% (*X*^2^=54.29, *P*\<.001), 98.3% (*X*^2^=362.58, *P*\<.001) ([Figure 2-F](#F2){ref-type="fig"}), and sROC AUC was 0.82 (SE=0.05) while Q^\*^value was 0.75 (SE=0.05)([Figure 3-C](#F3){ref-type="fig"}).

![Diagnosis test accuracy of pressure ulcer risk assessment scales](IJPH-45-122-g002){#F2}

![Summary receiver operating characteristic curve](IJPH-45-122-g003){#F3}

Through detailed analytical research, the total sensitivity of 60s was 0.69 (95% CI 0.61, 0.76) with a cut-off point of 94.2% (*X*^2^=34.75, *P*\<.001) In Standard reference AHCPR ([@B23], [@B30]--[@B31]), total sensitivity was 0.61 (95% CI 0.52, 0.70), and heterogeneity between literature studies was 94.5% (*X*^2^=36.17, *P*\<.001). The total sensitivity of NPUAP was 0.82 (95% CI 0.69, 0.91) and the heterogeneity between literature studies was 0.0% (*X*^2^=0.40, *P*=.53). In all cases of total specificity, the heterogeneity between literature studies was over 90%.

Waterlow Scale
--------------

There were six literature studies ([@B30]--[@B31], [@B34]--[@B37]) applying the Waterlow scale, involving 1,268 inpatients. All literature studies were prospective studies in cases of no pressure ulcers of inpatients, and the subjects of study were elderly men over their 60s of average age. Standard reference was used in two literature studies with NPUAP ([@B36]--[@B37]) and AHCPR ([@B30]--[@B31]), and EPUAP ([@B35]), TDCPS ([@B34]) in one literature study. The cut-off point was applied variously; one literature study had 9 points ([@B30]), 15points ([@B37]), and 17points ([@B36]) and two literature studies had 16 points ([@B31], [@B34]). This cut-off point was not reported ([@B35]).

Total sensitivity was 0.55 (95% CI 0.49, 0.62) ([Figure 2-G](#F2){ref-type="fig"}), and total specificity was 0.82 (95% CI 0.80, 0.85) ([Figure 2-H](#F2){ref-type="fig"}). In addition, total sensitivity was 0.55 (95% CI 0.49, 0.62), between studies was high, as 90.0% (*X*^2^=49.76, *P*\<.001), 97.2% (*X*^2^=177.24, *P*\<.001), and sROC AUC was 0.82 (SE=0.03) while Q^\*^value was 0.75 (SE=0.03), regarding the heterogeneity, in the result of meta-analysis. Through detailed analytical research, in the case of standard reference NPUAP, total sensitivity was 0.68 (95%CI 0.43, 0.87) but heterogeneity between literature did not exist, as 0.0% (*X*^2^=0.05, *P*=0.83), while heterogeneity between studies of total sensitivity and specificity of standard reference AHCPR was high, over 80%. Modified Braden scale was used in six studies ([@B23], [@B32], [@B38]--[@B41]), and the Cubbin & Jackson scale was applied in four studies ([@B6], [@B40], [@B42]--[@B43]).

Modified Braden Scale
---------------------

Regarding the literature on the Modified Braden scale, four studies ([@B23], [@B39]--[@B41]) were reported in Korea and two studies ([@B32], [@B38]) in China. All literature studies are prospective studies aimed for patients who do not have pressure ulcers when they were in ward ([@B38]--[@B39]) and ICU ([@B23], [@B40]) which involve 1,314 patients. Three literature studies ([@B32], [@B39]--[@B40]) are about average age 50s, two literature studies about 60s, and one literature study about 70s ([@B38]). Three volumes are about IC; three volumes are about the study of standard reference AHCPR ([@B23], [@B40]--[@B41]); two volumes are about NPUAP ([@B32], [@B38]); one volume is about Bergstrom ([@B39]).

Total sensitivity was0.95 (95% CI 0.91, 0.98) and heterogeneity between literature studies was moderate, as 49.9% (*X*^2^=9.99, *P=0*.08) ([Figure 2-C](#F2){ref-type="fig"}). Total specificity was 0.70 (95% CI 0.68, 0.73) and heterogeneity between literature studies was high, as 96.4% (*X*^2^=138.48, *P*\<.001) ([Figure 2-D](#F2){ref-type="fig"}). sROC AUC was 0.93 (SE=0.03); Q^\*^value was 0.87 (SE=0.03) ([Figure 3-B](#F3){ref-type="fig"}).

Through detailed analytical research, In the case of 50s, the total sensitivity was 0.92 (95% CI 0.83, 0.97), and heterogeneity between literature studies was 0.0% (*X*^2^=1.04, *P=*.60). The total sensitivity of 60s was 1.00 (95% CI 0.95, 1.00) while heterogeneity between literature studies was 0.0% (*X*^2^=0.00, *P*=1.00). The total sensitivity of standard reference was 0.98 (95% CI 0.94, 1.00), and heterogeneity between literature studies was 50.9% (*X*^2^=4.07,*P*=.13) while NPUAP is only used in literature of China. Total specificity was over 90% in every case.

Cubbin & Jackson Scale
----------------------

Literature on the Cubbin & Jackson Scale was four volumes ([@B6], [@B40], [@B42]--[@B43]), conducted in Korea with 662 ICU patients as a prospective study. Except Jun exec ([@B42]), studies were conducted with patients who did not have pressure sores, and the average age of patients was in their 50s ([@B40], [@B43]) and 60s ([@B6], [@B42]). There were two literature studies ([@B40], [@B42]) with standard reference AHCPR, and one study ([@B6]) with NPUAP, one volume ([@B43]) on Lowthian. The cut-off point was used in two volumes ([@B6], [@B42]) in 24 points and 28 points ([@B40]) and 26 points ([@B43]).

The total sensitivity was 0.67 (95% CI 0.60, 0.74) ([Fig. 2-A](#F2){ref-type="fig"}), and the total specificity was 0.75 (95% CI 0.70, 0.79) ([Fig. 2-B](#F2){ref-type="fig"}). Each heterogeneity between literature studies was high, as 92.8% (*X*^2^=41.54, *P*\<.001), 82.0% (*X*^2^=16.70, *P=.*001). sROC AUC was 0.82 (SE=0.06) and Q^\*^value was 0.75 (SE=0.05) ([Fig.3-A](#F3){ref-type="fig"}).

Through detailed analytical research, despite division of 50s and 60s, the heterogeneity was high, over 90%; from detailed analysis of two literature studies that used standard reference AHCPR, the total sensitivity was 0.92(95% CI0.83, 0.97) and the heterogeneity between literature studies was5.4% (*X*^2^=1.06, *P*=.30); the total specificity was 0.75 (95% CI 0.70, 0.81) and the heterogeneity between literature studies was 91.4% (*X*^2^=11.64, *P*=.001).

Comparison of Norton scale and Waterlow scale
---------------------------------------------

Two pressure ulcer risk scales were suggested in six literature studies out of 17 included in this study. Three literature studies are about comparison of Norton scale and Waterlow scale with the same subjects ([@B30]--[@B31], [@B34]). There were two studies ([@B23], [@B32]) comparing Norton scale and modified Braden, and only one study ([@B40]) comparing modified Braden and Cubbin & Jackson scale. In the results of detailed analysis, Norton scale had 0.53 (95% CI 0.43, 0.63) total sensitivity and 81.4% (*X*^2^=10.73, *P*=.005) heterogeneity between literature studies, while total specificity was 0.88 (95% CI 0.83, 0.91) and heterogeneity between literature studies was 97.9% (*X*^2^=93.06, *P*\<.001); sROC AUC was 0.81 (SE=0.06) and Q^\*^value was 0.75 (SE=0.05). Waterlow scale had 0.74 (95%CI 0.64, 0.82) total sensitivity and 88.4% (*X*^2^=17.25, *P*=.001) heterogeneity between literature studies while total specificity was 0.66 (95% CI 0.60, 0.72) and heterogeneity between literature studies was 95.1% (*X*^2^=40.54, *P*\<.001); sROC AUC was 0.82 (SE=0.03) and Q^\*^ value was 0.76 (SE=0.03) ([Table 2](#T2){ref-type="table"}--[3](#T3){ref-type="table"}).

###### 

Characteristics of selected studies

  **Year of publication**      **Authors**           **Location**   **Types of studies**   **Participants**   **Reference standards**   **Cut off point**   **2×2 Table**   **Value (95% Confidence interval)**                                                                                                                                       
  ---------------------------- --------------------- -------------- ---------------------- ------------------ ------------------------- ------------------- --------------- ------------------------------------- ------------------ ------ ----- ----- ---- ----- ------------------- ------------------- ---------------------- ------------------- -----------------------
                               **Norton scale**                                                                                                                                                                                                                                                                                                       
  2005                         Defloor & Grypdonck   Belgium        Pro                    LTCF               PUs (−)                   84.6± 7.9           369:1,40 3      1,772                                 EPUAP (1999)       \<14   152   710   35   875   0.81 (0.75--0.86)   0.55 (0.55--0.56)   1.82 (1.65--1.95)      0.34 (0.25--0.46)   5.35 (3.60--7.98)
  2005                         Jalali & Rezaie       Iran           Pro                    ICU & wards        PUs (−)                   60.0                100:130         230                                   AHCPR (1994)       \<16   36    0     38   156   0.49 (0.43--0.49)   1.00 (0.89--1.00)   \-                     0.51 (0.51--0.58)   \-
  2005                         Kwong et al.          China          Prospective            \-                 PUs (−)                   54.1± 16.9          253:176         429                                   NPUAP (1989)       \<14   8     164   1    256   0.89 (0.51--0.99)   0.61 (0.60--0.61)   2.28 (1.28--2.56)      0.18 (0.01--0.81)   12.49 (1.57--268.73)
  2003                         Lee at al.            Korea          Pro                    ICU                PUs (−)                   62.0                64:48           112                                   AHCPR (1994)       \<16   34    55    1    22    0.97 (0.86--1.00)   0.29 (0.23--0.30)   1.36 (1.12--1.42)      0.10 (0.01--0.62)   13.60 (1.80--283.14)
  2003                         Marrie et al.         Canada         Retro                  \-                 \-                        61.0± 18.0          98:90           188                                   NPUAP (2002)       ≤14    37    121   9    21    0.80 (0.69--0.90)   0.15 (0.11--0.18)   0.94 (0.78--1.09)      1.32 (0.59--2.78)   0.71 (0.28--1.85)
  2001                         Charlier              Australia      Prospective            Hosp.              PUs (−)                   \-                  29:33           62                                    AHCPR (1995)       \<15   3     1     8    50    0.27 (0.08--0.36)   0.98 (0.94--1.00)   13.91 (1.38--344.42)   0.74 (0.64--0.98)   18.75 (1.42--536.60)
  1998                         Pang & Wong           Hong Kong      Pro                    Hosp.              PUs (−)                   45--92              52:54           106                                   TDCPS (1983)       \<16   17    35    4    50    0.81 (0.60--0.94)   0.59 (0.54--0.62)   1.97 (1.29--2.46)      0.32 (0.10--0.75)   6.07 (1.71--23.53)
                               **Waterlow scale**                                                                                                                                                                                                                                                                                                     
  2010                         Webster et al.        Australia      Pro                    Ward               PUs (−)                   65.3± 7.7           137:137         274                                   NPUAP (2007)       \<15   8     55    4    207   0.67 (0.36--0.89)   0.79 (0.78--0.80)   3.18 (1.61--4.43)      0.42 (0.14--0.82)   7.53 (1.96--31.04)
  2009                         Serpa et al.          Brazil         Pro                    ICU & wards        PUs (−)                   71.1± 5.5           49:49           98                                    NPUAP (2007)       \<17   5     30    2    61    0.71 (0.31--0.95)   0.67 (0.64--0.69)   2.17 (0.86--3.04)      0.43 (0.08--1.08)   5.08 (0.80--40.52)
  2008                         Compton et al.        Germany        Pro                    ICU                PUs (−)                   66.0                392:306         698                                   EPUAP (1999)              45    31    76   546   0.37 (0.31--0.43)   0.95 (0.93--0.96)   6.9 (4.50--10.66)      0.66 (0.59--0.75)   10.4 (6.03--18.08)
  2005                         Jalali & Rezaie       Iran           Pro                    ICU & wards        PUs (−)                   60.0                100:130         230                                   AHCPR (1994)       \<16   47    28    27   128   0.64 (0.54--0.72)   0.82 (0.78--0.86)   3.54 (2.41--5.13)      0.45 (0.33--0.59)   7.96 (4.07--15.67)
  2001                         Charlier              Australia      Prospective            Hosp.              PUs (−)                   --                  29:33           62                                    AHCPR (1995)       \<9    11    23    0    28    1.00 (0.71--1.00)   0.55 (0.49--0.55)   2.22 (1.37--2.22)      0.00 (0.00--0.60)   \-
  1998                         Pang & Wong           Hong Kong      Pro                    Hosp.              PUs (−)                   45--92              52:54           106                                   TDCPS (1983)       \<16   20    48    1    37    0.95 (0.76--1.00)   0.44 (0.39--0.45)   1.69 (1.24--1.80)      0.11 (0.01--0.62)   15.42 (2.02--322.34)
  **Modified Braden scale**                                                                                                                                                                                                                                                                                                                           
  *By Choi & Song*                                                                                                                                                                                                                                                                                                                                    
  2009                         Kim et al.            Korea          Prospective            SICU               PUs (−)                   58.1± 1.2           145:74          219                                   AHCPR (1994)       \<21   38    55    2    124   0.95 (0.83--0.99)   0.69 (0.67--0.70)   3.09 (2.48--3.33)      0.07 (0.01--0.26)   42.84 (9.59--266.55)
  2004                         Kim & Choi            Korea          Prospective            ICU & ward         PUs (−)                   60.7                \-              211                                   AHCPR (1996)       \<23   34    42    0    135   1.00 (0.88--1.00)   0.76 (0.74--0.76)   4.21 (3.39--4.21)      0.00 (0.00--0.16)   \-
  2003                         Lee at al.            Korea          Prospective            ICU                PUs (−)                   62.0                64:48           112                                   AHCPR (1996)       \<24   35    63    0    14    1.00 (0.90--1.00)   0.18 (0.14--0.18)   1.22 (1.04--1.22)      0.00 (0.00--0.73)   \-
  1991                         Choi & Song           Korea          Prospective            Wards              PUs (−)                   54.1                89:57           146                                   Bergstrom (1987)   \<24   14    11    2    119   0.88 (0.64--0.98)   0.92 (0.89--0.93)   10.34 (5.57--13.58)    0.14 (0.02--0.41)   75.73 (13.54--559.83)
  ***By Pang & Wong***                                                                                                                                                                                                                                                                                                                                
  2009                         Chan et al.           Hong Kong      Prospective            OS ward            PUs (−)                   79.4± 10.9          30:167          197                                   NPUAP (2007)       \<19   16    68    2    111   0.89 (0.65--0.98)   0.62 (0.60--0.63)   2.34 (1.61--2.65)      0.18 (0.03--0.59)   13.06 (2.75--84.99)
  2005                         Kwong et al.          China          Prospective            \-                 PUs (−)                   54.1± 16.9          253:176         429                                   NPUAP (1989)       \<14   8     105   1    315   0.89 (0.51--0.99)   0.75 (0.74--0.75)   3.56 (1.98--4.01)      0.15 (0.01--0.66)   24.00 (3.01--517.73)
  **Cubbin & Jackson scale**                                                                                                                                                                                                                                                                                                                          
  2009                         Kim et al.            Korea          Prospective            SICU               PUs (−)                   58.1± 1.2           145:74          219                                   AHCPR (1994)       \<28   38    33    2    146   0.95 (0.83--0.99)   0.82 (0.79--0.83)   5.15 (3.94--5.66)      0.06 (0.01--0.21)   84.06 (18.39--531.62)
  2006                         Im & Park             Korea          Prospective            ICU                PUs (−)                   62.1± 1.8           43:35           78                                    NPUAP (1989)       \<24   9     8     13   48    0.41 (0.24--0.57)   0.86 (0.79--0.92)   2.86 (1.13--7.07)      0.69 (0.47--0.97)   4.15 (1.17--15.06)
  2003                         Jun et al.            Korea          Prospective            ICU                PUs (±)                   62.0                64:48           112                                   AHCPR (1994)       \<24   31    30    4    47    0.89 (0.75--0.96)   0.61 (0.55--0.65)   2.27 (1.65--2.71)      0.19 (0.06--0.46)   12.14 (3.58--45.36)
  1997                         Kim et al.            Korea          Prospective            ICU                PUs (−)                   56.2                162:91          253                                   Lowthian (1989)    \<26   52    45    45   111   0.54 (0.46--0.61)   0.71 (0.66--0.76)   1.86 (1.34--2.55)      0.65 (0.51--0.83)   2.85 (1.63--5.01)

Notes: PUs= Pressure ulcers; TP= True positive; FP= False positive; FN= False negative; TN= True negative; SN= Sensitivity; SP= Specificity; PLR= Positive likelihood ratio; NLR= Negative likelihood ratio; DOR= Diagnosis odds ratio; Retro= Retrospective study; Hosp.= Hospital; ICD-9= International Classification of Diseases version 9; Pro= Prospective study; LTCF= Long term care facilities; NPUAP= National Pressure Ulcer Advisory Panel; EPUAP= European Pressure Ulcer Advisory Panel; AHCPR= Agency for Health Care Policy and Research; TDCPS= Torrence Developmental Classification of Pressure Sore.

###### 

Summary results of Meta-analysis by four risk assessment tools

  **Scale**            **Study No.**   **Sensitivity (95% CI)**   **Specificity (95% CI)**   **Positive LR (95% CI)**   **Negative LR (95% CI)**   **DOR (95% CI)**        **AUC (SE)**   **Q^\*^ (SE)**
  -------------------- --------------- -------------------------- -------------------------- -------------------------- -------------------------- ----------------------- -------------- ----------------
  Cubbin & Jackson     4               0.67 (0.60--0.74)          0.75 (0.71, 0.79)          2.80 (1.66--4.72)          0.34 (0.15, 0.76)          9.46 (2.41, 37.22)      0.82 (0.06)    0.75 (0.06)
  Modified Braden      6               0.95 (0.91--0.98)          0.70 (0.68, 0.73)          3.23 (1.74--6.00)          0.11 (0.06, 0.21)          35.16 (16.62, 74.37)    0.93 (0.03)    0.87 (0.03)
    by Choi & Song     4               0.97 (0.92, 0.99)          0.70 (0.66, 0.73)          3.47 (1.33, 9.06)          0.08 (0.04, 0.19)          56.56 (21.88, 146.21)   0.95 (0.02)    0.90 (0.03)
    by Pang & Wong     2               0.89 (0.71, 0.98)          0.71 (0.67, 0.75)          2.87 (1.88, 4.38)          0.17 (0.06, 0.49)          16.06 (4.75, 54.35)     \-             \-
  Norton               7               0.75 (0.70--0.79)          0.57 (0.55, 0.59)          1.77 (1.26, 2.50)          0.49 (0.32--0.76)          7.57 (2.53--22.64)      0.82 (0.05)    0.75 (0.04)
  Waterlow             6               0.55 (0.49--0.62)          0.82 (0.80, 0.85)          2.89 (1.74, 4.79)          0.46 (0.31--0.70)          9.22 (6.43--13.23)      0.82 (0.03)    0.75 (0.03)
  Sub-group Analysis   3                                                                                                                                                                  
    Norton vs                          0.53 (0.43, 0.63)          0.88 (0.83, 0.91)          14.87 (0.12, 1805.61)      0.56 (0.39,, 0.80)         25.67 (2.28, 289.21)    0.81 (0.06)    0.75 (0.05)
    Waterlow                           0.74 (0.64, 0.82)          0.66 (0.60, 0.72)          2.30 (1.37, 3.83)          0.23 (0.06, 0.87)          8.84 (4.92, 15.89)      0.82 (0.03)    0.76 (0.03)

Note: LR=Likelihood ration; DOR= Diagnostic odds ratio; AUC= Area under the curve; SE= Standard error.

Discussion
==========

In this experiment, we conducted a meta-analysis for selection of an appropriate pressure ulcer risk scale for patients; to examine the validity of Norton, Waterlow, Modified Braden, and Cubbin & Jackson tools, we analyzed 17 volumes and 5,185 diagnostic results of patients. Predictable validity is standard as we can see from an examination and evaluation tool that guides estimation of future results. The standard validity is comprised of sensitivity, specialty, positive and negative predictability ([@B44]). High sensitivity leads to low specialty, which is useful in measurement of the positive and negative predictability but not for precise meta-analysis because it influences the prevalence rate ([@B24]). To compensate for the defect, we can use the ROC analysis method for the calculation of general validity. ROC analysis method is a widely used method for management of not only a medical diagnosis and standard for decision-making but also a tool for development and standardization ([@B45]).

According to meta-analysis, the most sensitive tool is the modified Braden and is Waterlow, which is the tool with the greatest specialty. All pressure ulcer risk scales except the modified Braden, which have the same sROC AUC. Therefore, each tool has moderate accuracy for predictable validity of pressure sores of approximately 0.7 \<AUC ≤0.9. Only the Waterlow tool was interpreted as insufficient because pressure ulcer assessment is a screening inspection that prefers a sensitive tool rather than a specialized tool. However, after a detailed analysis of the Norton scale and Waterlow scale, the Waterlow scale has a higher sensitivity (0.53 versus 0.74) and low specialty (0.88 versus 0.66) while they both have similar measured value in the ROC analysis method.

In order to analyze the problem in the studies, we classified them as average life expectancy, locations of hospitalization, and reference average standard. Four tools tend to lose difference between studies of combined sensitivity in common, depending on standard reference. To interpret the result of the research, using a skin assessment tool of consistent standardized factors of pressure ulcers in the clinic will increase the discrimination of pressure ulcer risk scale. Not all differences between studies were decreased, except the modified Braden's tool in patients' age. However, it is shown as a phenomenon of decreasing discrimination, because fewer studies at a given age stage might induce analysis in the higher proportion range of age, or no difference was shown from the late 50s to early 60s, despite the division of age. At places of hospitalization, only ICU is targeted in use of Cabbin & Jackson's tool, and modified Braden's tool can only be analyzed in depth, because Norton and Waterlow's tools are for inpatients. In the result of division: ICU and ward, we can see the decreasing aspect of difference between studies. To sum up, application in specific groups, which can well reflect the properties of each tool, will increase predictive validity, because pressure ulcer risk scales are easily affected by age, locations of hospitalization, and standard reference.

This study uses original cut-off points. Optimized cut-off point between studies does not show consistency in each instrument, and transition width is wide depending on age group. To interpret the result, it is misleading to determine and use the cutoff points of the current pressure ulcer risk scale, and applying with category would be more appropriate than specific threshold.

In other words, Norton, Waterlow, Modified Braden and Cubbin & Jackson scale cannot show consistent measured value, and they cannot determine the risk of pressure sore outbreak because of heterogeneity between studies, thus they are not suitable for use as general tools in the clinic toward inpatients. This statement is in agreement with studies of Anthony exec ([@B46]) in that current pressure ulcer risk scales do not predict the outbreak of pressure ulcers, and some components include unnecessary items, it is apart from correlation of pressure sore outbreak. Only when we think about the alternative that can be applied instantly at this important time of prevention of pressure ulcers, use of the Norton scale first, which can be used more easily in the clinic, would be an effective method if the predictive validities of the four scales are similar, and then use the Waterlow scale with high specificity to increase the accuracy of the current pressure risk assessment.

Conclusion
==========

In this research, predictive validity of Waterlow, Modified Braden, and Cubbin & Jackson's pressure ulcer risk scales was determined by meta-analysis based on 17 well-designed studies assessing diagnosis. As sROC AUC is over 0.7, all analyzed instruments show moderate predictive validity, but they have limited interpretation due to high differences between studies. In addition, Waterlow's instrument is insufficient for use as a screening tool owing to low sensitivity. Pressure ulcer risk scales in current use are not suitable for uniform practice on inpatients under standardized criteria. Therefore, in order to provide more effective nursing care of pressure ulcers, not only in-depth analysis reflecting patients' characteristics is required, but also a new or modified pressure ulcer risk scale should be developed with complementation of strength and weakness in existing tools.

Ethical Considerations
======================

Ethical issues (Including plagiarism, Informed Consent, misconduct, data fabrication and/or falsification, double publication and/or submission, redundancy, etc) have been completely observed by the authors.

This work was supported by the Soonchunhyang University Research Fund. Besides, it was supported by Hannam University, Academic Research Fund of 2016. The authors declare that there is no conflict of interests.
