Development and evaluation of an instrument for the critical appraisal of randomized controlled trials of natural products Physical rehabilitation interventions for adult patients with critical illness across the continuum of recovery: an overview of systematic reviews protocol AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Development and validation of the Liverpool infant bronchiolitis severity score: a research protocol. Establishing the minimal number of items for a responsive, valid, health-related quality of life instrument. Bedside assessment of executive cognitive impairment: the executive interview. Feasibility, psychometric performance, and stability across modes of administration of the CARES-SF. The Patient Health Questionnaire-9 for detection of major depressive disorder in primary care: consequences of current thresholds in a crosssectional study Comparison of Checklist and Anchored Global Rating Instruments for Performance Rating of Simulated Pediatric Emergencies The Cannabis Use Problems Identification Test (CUPIT): development, reliability, concurrent and predictive validity among adolescents and adults. The Behavior Problems Inventory: An Instrument for the Assessment of Self-Injury, Stereotyped Behavior, and Aggression/Destruction in Individuals with Developmental Disabilities Validation of the WHO Quality of Life assessment instrument (WHOQOL-100) in a population of Dutch adult psychiatric outpatients. Psychometric Comparison of the PROMIS Physical Function CAT With the FAAM and FFI for Measuring Patient-Reported Outcomes Reliability of two instruments for critical assessment of economic evaluations. Development and Validation of a Novel Motor-Cognitive Assessment Strategy of Compensatory Sit-to-Stand Maneuvers in People With Dementia Fall risk assessment: a prospective investigation of nurses' clinical judgement and risk assessment tools in predicting patient falls. Evaluation of delirium in critically ill patients: Validation of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) The Brown Assessment of Beliefs Scale: reliability and validity. Validity and Reliability of the Eating Assessment Tool (EAT-10) Factor Structure, Reliability and Measurement Invariance of the Alberta Context Tool and the Conceptual Research Utilization Scale, for German Residential Long Term Care The Liebowitz social anxiety scale for children and adolescents: an initial psychometric investigation.
Background The efficacy of natural products ( NPs ) is being evaluated using r and omized controlled trials ( RCTs ) with increasing frequency , yet a search of the literature did not identify a widely accepted critical appraisal instrument developed specifically for use with NPs . The purpose of this project was to develop and evaluate a critical appraisal instrument that is sufficiently rigorous to be used in evaluating RCTs of conventional medicines , and also has a section specific for use with single entity NPs , including herbs and natural source d chemicals . Methods Three phases of the project included : 1 ) using experts and a Delphi process to reach consensus on a list of items essential in describing the identity of an NP ; 2 ) compiling a list of non-NP items important for evaluating the quality of an RCT using systematic review methodology to identify published instruments and then compiling item categories that were part of a vali date d instrument and /or had empirical evidence to support their inclusion and 3 ) conducting a field test to compare the new instrument to a published instrument for usefulness in evaluating the quality of 3 RCTs of a NP and in applying results to practice . Results Two Delphi rounds result ed in a list of 15 items essential in describing NPs . Seventeen item categories fitting inclusion criteria were identified from published instruments for conventional medicines . The new assessment instrument was assembled based on content of the two lists and the addition of a Review er 's Conclusion section . The field test of the new instrument showed good criterion validity . Participants found it useful in translating evidence from RCTs to practice . Conclusion A new instrument for the critical appraisal of RCTs of NPs was developed and tested . The instrument is distinct from other available assessment instruments for RCTs of NPs in its systematic development and validation . The instrument is ready to be used by pharmacy students , health care practitioners and academics and will continue to be refined as required Background Patients admitted to the intensive care unit with critical illness often experience significant physical impairments , which typically persist for many years following resolution of the original illness . Physical rehabilitation interventions that enhance restoration of physical function have been evaluated across the continuum of recovery following critical illness including within the intensive care unit , following discharge to the ward and beyond hospital discharge . Multiple systematic review s have been published appraising the exp and ing evidence investigating these physical rehabilitation interventions , although there appears to be variability in review methodology and quality . We aim to conduct an overview of existing systematic review s of physical rehabilitation interventions for adult intensive care patients across the continuum of recovery . Methods / design This protocol has been developed according to the Preferred Reporting Items for Systematic Review s and Meta-Analyses Protocol ( PRISMA -P ) guidelines . We will search the Cochrane Systematic Review Data base , Data base of Abstract s of Review s of Effectiveness , Cochrane Central Register of Controlled Trials , MEDLINE , Excerpta Medica Data base and Cumulative Index to Nursing and Allied Health Literature data bases . We will include systematic review s of r and omised controlled trials of adult patients , admitted to the intensive care unit and who have received physical rehabilitation interventions at any time point during their recovery . Data extraction will include systematic review aims and rationale , study types , population s , interventions , comparators , outcomes and quality appraisal method . Primary outcomes of interest will focus on findings reflecting recovery of physical function . Quality of reporting and method ological quality will be appraised using the PRISMA checklist and the Assessment of Multiple Systematic Review s tool . Discussion We anticipate the findings from this novel overview of systematic review s will contribute to the synthesis and interpretation of existing evidence regarding physical rehabilitation interventions and physical recovery in post-critical illness patients across the continuum of recovery . Systematic review registration PROSPERO CRD42015001068 OBJECTIVE Our purpose was to measure the agreement , reliability , construct validity , and feasibility of a measurement tool to assess  systematic review s ( AMSTAR ) . STUDY DESIGN AND SETTING We r and omly selected 30 systematic review s from a data base . Each was assessed by two review ers using : ( 1 ) the enhanced quality assessment question naire ( Overview of Quality Assessment Question naire [ OQAQ ] ) ; ( 2 ) Sacks ' instrument ; and ( 3 ) our newly developed measurement tool ( AMSTAR ) . We report on reliability ( interobserver kappas of the 11 AMSTAR items ) , intraclass correlation coefficients ( ICCs ) of the sum scores , construct validity ( ICCs of the sum scores of AMSTAR compared with those of other instruments ) , and completion times . RESULTS The interrater agreement of the individual items of AMSTAR was substantial with a mean kappa of 0.70 ( 95 % confidence interval [ CI ] : 0.57 , 0.83 ) ( range : 0.38 - 1.0 ) . Kappas recorded for the other instruments were 0.63 ( 95 % CI : 0.38 , 0.78 ) for enhanced OQAQ and 0.40 ( 95 % CI : 0.29 , 0.50 ) for the Sacks ' instrument . The ICC of the total score for AMSTAR was 0.84 ( 95 % CI : 0.65 , 0.92 ) compared with 0.91 ( 95 % CI : 0.82 , 0.96 ) for OQAQ and 0.86 ( 95 % CI : 0.71 , 0.94 ) for the Sacks ' instrument . AMSTAR proved easy to apply , each review taking about 15 minutes to complete . CONCLUSIONS AMSTAR has good agreement , reliability , construct validity , and feasibility . These findings need confirmation by a broader range of assessors and a more diverse range of review AIM To develop and vali date a  bronchiolitis severity scoring instrument  for use by nurses and other healthcare professions . BACKGROUND Bronchiolitis is a viral lower respiratory tract infection of infancy . In industrialized countries , admission rates have increased over the last decade with up to 3 % of all infants born being admitted to hospital . A small number of these hospitalized infants will require admission to critical care for either invasive or non-invasive ventilation . During the seasonal epidemic , the number of unplanned admissions to critical care with bronchiolitis substantially increases . DESIGN We will use a mixed methods study design . METHODS We will use scale development and psychometric methods to develop a scoring instrument and to test the instrument for content , construct and criterion validity and reliability in several different clinical locations . This study protocol has been review ed and approved by the NHS National Research Ethics Service , January 2011 . DISCUSSION There is an urgent need to develop a valid and reliable severity scoring instrument sensitive to clinical changes in the infant , to facilitate clinical decision-making and help st and ardize patient care . Furthermore , a valid and reliable scoring instrument could also be used as a proxy patient-reported outcome measure to evaluate the efficacy of clinical interventions in r and omized controlled trials Reducing the number of items in a health-related quality of life instrument will enhance efficiency . However , it is important to maintain measurement properties . We determined the effect of reducing items from each domain ( dyspnea , fatigue , emotion and mastery ) of the 20-item Chronic Respiratory Question naire ( CRQ ) .  Three r and omized trials of respiratory rehabilitation provided data . We removed items one at a time from each domain in three orders : by item impact , item responsiveness , and at r and om . Responsiveness , test-retest reliability and construct validity were evaluated at each step . Responsiveness and reliability , evaluated by intraclass correlation coefficients ( ICC ) , were reduced marginally as the number of items was reduced to two items per domain . The deterioration was greatest when reducing from two items to one . To detect a particular effect , sample size would increase by about 10 % when reducing the number of items in a domain to 2 . Construct validity showed a more marked deterioration . Reducing to two items per domain would maintain responsiveness and reliability of the CRQ at an acceptable level , with a trade-off of reduced construct validity and increase in sample size requirements OBJECTIVE This study is a pilot validation of the  Executive Interview ( EXIT ) , a novel instrument design ed to assess executive cognitive function ( ECF ) at the bedside . DESIGN Inter-rater reliability testing and validation using inter-group comparisons across levels of care and measures of cognition and behavior . PARTICIPANTS Forty elderly subjects r and omly selected across four levels of care . SETTING Setting s ranged from independent living apartments to design ated Alzheimer 's Special Care units in a single 537-bed retirement community . MEASUREMENTS The EXIT : a 10-minute , 25-item interview scored from 0 - 50 ( higher scores = greater executive dyscontrol ) was administered by a physician . Subjects were also administered the  Mini-Mental State Exam ( MMSE ) and traditional tests of " frontal " executive function by a neuropsychologist , and the Nursing Home Behavior Problem Scale ( NHBPS ) by Licensed Vocational Nurses . RESULTS Interrater reliability was high ( r = .90 ) . EXIT scores correlated well with other measures of ECF . The interview discriminated among residents at each level of care . In contrast , the MMSE did not discriminate apartment-dwelling from residential care residents , or residential care from nursing home residents . The EXIT was highly correlated with disruptive behaviors as measured by the NHBPS ( r = .79 ) . CONCLUSIONS These preliminary findings suggest that the EXIT is a valid and reliable instrument for the assessment of executive impairment at the bedside . It correlates well with level of care and problem behavior . It discriminates residents at earlier stages of cognitive impairment than the MMSE BACKGROUND The primary objective was to investigate the validity and reliability of the Cancer Rehabilitation Evaluation System-Short Form ( the CARES-SF ) as a quality of life instrument in clinical trials [ 10 ] . PATIENTS AND METHODS A heterogeneous sample of 485 cancer patients completed the CARES-SF before treatment ( T1 ) , one month later ( T2 ) , and three months following T2 ( T3 ) . At T3 the patients completed the question naire either by mail , in a telephone interview , or in the clinic . A sub- sample of patients completed the CARES-SF a fourth time ( T4 ) one week following T3 , for purpose s of test-retest reliability estimation . RESULTS On average , the CARES-SF required 11 minutes for completion and could be completed by 90 % of the patients without assistance . However , 82 % of the patients reported difficulty with at least one item . Multitrait scaling analysis and factor analysis generally confirmed the hypothesized scale structure . Internal consistency reliability coefficients exceeded the 0.70 criterion for four of six multi-item scales . The test-retest reliability coefficients of the six scales were above 0.70 . No systematic differences were found in the psychometrics of the CARES-SF across administration conditions . In the mail condition the proportion of missing items was significantly higher , and patients reported having significantly more problems than in both the telephone and in-clinic condition . According to expectation , selective scales distinguished clearly between patients differing in disease stage , performance status , treatment modality and tumor response . Additionally , selective scales were responsive to changes in health status over time . CONCLUSION These results lend support to the reliability and validity of the CARES-SF in assessing the quality of life of patients with cancer . At the same time , efforts to refine the question naire are recommended Background There is a need for brief instruments to ascertain the diagnosis of major depressive disorder . In this study , we present the reliability , construct validity and accuracy of the PHQ-9 and PHQ-2 to detect  major depressive disorder in primary care . Methods Cross-sectional analyses within a large prospect i ve cohort study ( PREDICT-NL ) . Data was collected in seven large general practice s in the centre of the Netherl and s. 1338 subjects were recruited in the general practice waiting room , irrespective of their presenting complaint . The diagnostic accuracy ( the area under the ROC curve and sensitivities and specificities for various thresholds ) was calculated against a diagnosis of major depressive disorder determined with the Composite International Diagnostic Interview ( CIDI ) . Results The PHQ-9 showed a high degree of internal consistency ( ICC = 0.88 ) and test-retest reliability ( correlation = 0.94 ) . With respect to construct validity , it showed a clear association with functional status measurements , sick days and number of consultations . The discriminative ability was good for the  PHQ-9  ( area under the ROC curve = 0.87 , 95 % CI : 0.84 - 0.90 ) and the PHQ-2 ( ROC area = 0.83 , 95 % CI 0.80 - 0.87 ) . Sensitivities at the recommended thresholds were 0.49 for the PHQ-9 at a score of 10 and 0.28 for a categorical algorithm . Adjustment of the threshold and the algorithm improved sensitivities to 0.82 and 0.84 respectively but the specificity decreased from 0.95 to 0.82 ( threshold ) and from 0.98 to 0.81 ( algorithm ) . Similar results were found for the  PHQ-2 : the recommended threshold of 3 had a sensitivity of 0.42 and lowering the threshold result ed in an improved sensitivity of 0.81 . Conclusion The PHQ-9 and the PHQ-2 are useful instruments to detect major depressive disorder in primary care , provided a high score is followed by an additional diagnostic work-up . However , often recommended thresholds for the PHQ-9 and the PHQ-2 result ed in many undetected major depressive disorders Purpose : To compare the psychometric performance of two rating instruments used to assess trainee performance in three clinical scenarios . Methods : This study was part of a two-phase , r and omized trial with a wait-list control condition assessing the effectiveness of a pediatric emergency medicine curriculum targeting general emergency medicine residents . Residents received 6 hours of instruction either before or after the first assessment .  Separate pairs of raters completed either a dichotomous checklist for each of three cases or the  Global Performance Assessment Tool ( GPAT )  , an anchored multidimensional scale . A fully crossed person × rater × case generalizability study was conducted . The effect of training year on performance is assessed using multivariate analysis of variance . Results : The person and person × case components accounted for most of the score variance for both instruments . Using either instrument , scores demonstrated a small but significant increase as training level increased when analyzed using a multivariate analysis of variance . The inter-rater reliability coefficient was > 0.9 for both instruments . Conclusions : We demonstrate that our checklist and anchored global rating instrument performed in a psychometrically similar fashion with high reliability . As long as proper attention is given to instrument design and testing and rater training , checklists and anchored assessment scales can produce reproducible data for a given population of subjects . The validity of the data arising for either instrument type must be assessed rigorously and with a focus , when practicable , on patient care outcomes AIMS To describe the empirical construction and initial validation of the Cannabis Use Problems Identification Test ( CUPIT ) , a brief self-report screening instrument for detection of currently and potentially problematic cannabis use . DESIGN In a three-phase prospect i ve design an item pool of c and i date questions was generated from a literature review and extensive expert consultation . The CUPIT internal structure , cross-sectional and longitudinal psychometric properties were then systematic ally tested among heterogeneous past-year users . PARTICIPANTS Volunteer participants were 212 high-risk adolescents ( n = 138 ) and adults ( n = 74 ) aged 13 - 61 years from multiple community setting s. MEASUREMENTS The comprehensive assessment battery included several established measures of cannabis-related pathology for CUPIT validation , with DSM-IV/ICD-10 diagnoses of cannabis use disorders as criterion st and ard . FINDINGS Sixteen items loading highly on two subscales derived from principal components analysis exhibited good to excellent test-retest ( 0.89 - 0.99 ) and internal consistency reliability ( 0.92 , 0.83 ) , and highly significant ability to discriminate diagnostic subgroups along the severity continuum ( non-problematic , risky , problematic use ) . Twelve months later , baseline CUPIT scores demonstrated highly significant longitudinal predictive utility for respondents ' follow-up diagnostic group membership . Receiver operating characteristic ( ROC ) analysis identified a  CUPIT score of 12 to be the optimal cut-point for maximizing sensitivity for both currently diagnosable cannabis use disorder and those at risk of meeting diagnostic criteria in the following 12 months . CONCLUSIONS The CUPIT is a brief cannabis screener that is reliable , valid and acceptable for use across diverse community setting s and consumers of all ages . The CUPIT has clear potential to assist with achievement of public health goals to reduce cannabis-related harms in the community The Behavior Problems Inventory ( BPI-01 ) is a 52-item respondent-based behavior rating instrument for self-injurious , stereotypic , and aggressive/destructive behavior in mental retardation and other developmental disabilities . Items are rated on a frequency scale and a severity scale . The BPI-01 was administered by interviewing direct care staff of 432 r and omly selected residents from a developmental center between the ages of 14 to 91 years . For 73 % of those selected , at least one problem was endorsed on the BPI-01 . A total of 43 % showed self-injury , 54 % stereotyped behavior , and 38 % aggressive/destructive behavior . Confirmatory factor analysis and item-total correlations supported the three a priori factors . Analyses of variance ( ANOVA ) showed that of the variables age , sex , and level of mental retardation , only the latter had a significant effect on the BPI-01 total score , the SIB subscale score , and the Stereotyped Behavior subscale score . Aggression/destruction was not significantly related to any of the three variables . Individuals with a diagnosis of pervasive developmental disorder had higher scores on all three subscales than those without , whereas residents with a diagnosis of stereotyped movement disorder had higher Stereotyped Behavior scale scores than those without . The BPI-01 was found to be a reliable ( retest reliability , internal consistency , and between-interviewer-agreement ) and valid ( factor and criterion validity ) behavior rating instrument for problem behaviors in mental retardation and developmental disabilities with a variety of potentially useful applications . Strengths and limitations of the instrument are discussed BACKGROUND Research concerning the psychometric properties of the  WHO Quality of Life Assessment Instrument ( WHOQOL-100 ) in general population s of psychiatric out patients has not been performed systematic ally . AIMS To examine the content validity , construct validity , and reliability of the WHOQOL-100 in a general population of Dutch adult psychiatric out patients . METHOD A total of 533 psychiatric out patients entered the study ( 438 r and omly selected , 85 internally referred ) . Participants completed self-administered question naires for measuring quality of life ( WHOQOL-100 ) , psychopathological symptoms ( SCL-90 ) , and perceived social support ( PSSS ) .  In addition , they underwent two semi-structured interviews in order to obtain Axis-I and Axis-II diagnoses , according to DSM-IV . RESULTS The drop-out percentage was low ( 7.1 % ) . Of the 24 facets of the WHOQOL-100 , 22 had a good distribution of scores , leaving out the facets physical environment and transport . Exploratory factor analysis revealed a four-factor structure , which was similar to earlier findings in patients with specific somatic diseases and depressive disorders . Various-a priori expected-positive and negative correlations were found between facets and domains of the WHOQOL-100 , and dimensions of the SCL-90 and the PSSS-score , indicating good construct validity of the WHOQOL-100 . The internal consistency of all facets and the four domains of the WHOQOL-100 was good ( Cronbach 's alpha 's ranging from 0.62 to 0.93 and 0.64 to 0.84 , respectively ) . Sparse and relatively low correlations were found between demographic characteristics ( age and sex ) and WHOQOL-100 scores . CONCLUSIONS Content validity , construct validity , and reliability of the WHOQOL-100 in a population of adult Dutch psychiatric out patients are good . The WHOQOL-100 appears to be a suitable instrument for measuring quality of life in adult psychiatric out patients Background : Selecting optimal patient-reported outcome ( PRO ) instruments is critical to improving the quality of health care . The purpose of this study was to compare the reliability , responsiveness , and efficiency of three PRO measures : the Foot and Ankle Ability Measure – Activity of Daily Living subscale ( FAAM_ADL ) , the Foot Function Index 5-point verbal rating scale ( FFI-5pt ) , and the PROMIS Physical Function computerized adaptive test ( PF CAT ) . Methods : Data were aggregated from 10 clinical sites in the AOFAS ’s National Orthopaedic Foot and Ankle Research ( OFAR ) Network from 311 patients who underwent elective surgery for a disorder of the foot or ankle . Patients were administered the FAAM_ADL , FFI-5pt , and PF CAT at their preoperative visit and at 6 months after surgery . Reliabilities were evaluated using a Rasch model . Responsiveness was calculated using paired sample s t test and efficiency was recorded as number of seconds to complete the instrument . Results : Similar reliabilities were found for the three instruments . Item reliabilities for FAAM_ADL , FFI-5pt , and PF CAT were all .99 . Pearson reliabilities for FAAM_ADL , FFI-5pt , and PF CAT were .95 , .93 , and .96 , respectively . On average , patients completed the FAAM_ADL in 179 seconds , the FFI-5pt in 194 seconds , and the PF CAT in 44 seconds , ( P < .001 ) . The PF CAT and FAAM_ADL showed significant improvement ( P = .01 and P = .001 , respectively ) in patients ’ physical function after treatment ; the FFI-5pt did not show improvement . Conclusions : Overall , the PF CAT performed best in terms of reliability , responsiveness , and efficiency in this broad sample of foot and ankle patients . It can be a potential replacement for the conventional PRO measures , but further validation is needed in conjunction with the PROMIS Pain instruments . Level of Evidence : Level I , prospect i ve comparative outcome study OBJECTIVE To assess the reliability of two instruments design ed for critical appraisal of economic evaluations : the Quality of Health Economic Studies ( QHES ) scale and the Pediatric Quality Appraisal Question naire ( PQAQ ) . METHODS Thirty published articles were chosen at r and om from a recent bibliography of economic evaluations in health promotion . The quality of each of these studies was assessed independently by two raters using each of the two instruments . Inter-rater reliability and the agreement between the instruments were measured using an intraclass correlation coefficient ( ICC ) . Cronbach 's generalizability theory was also used to assess the sources of variation in quality scores of the studies and to indicate where improvements in reliability could best be made . RESULTS Inter-rater reliability was excellent for both instruments ( ICC = 0.81 for the QHES and 0.80 for the PQAQ ) . Agreement between the instruments varied ( ICC = 0.77 for rater 1 and 0.56 for rater 2 ) . The biggest source of variation in the scores assigned to the articles was the quality of the study ( 56 % of total variance ) . Conventional measurement error explained 31 % of the total variance . Variation due to rater ( < 0.1 % ) and measurement instrument ( 1.8 % ) was very low . CONCLUSIONS The results suggest that the two instruments perform equally well . Choice of instrument can therefore be based on other criteria --simplicity and speed of application in the case of one , and detail in the information provided in the case of the other . There is little improvement in reliability to be gained from using more than one rater or more than one assessment of quality Background and Purpose : People with dementia show disease-specific sit-to-st and ( STS ) movement disorders , which relate to deficits of integrating cognitive aspects of motor processes into motor action organization . During STS training in rehabilitation therapy , compensatory STS movement maneuvers are taught aim ing to improve patients ' STS ability . Previous clinical STS measures do not address these maneuvers or assess cognitive aspects of their motor action organization . The purpose of this study was to develop and vali date a motor-cognitive STS assessment instrument for people with dementia ( Assessment of Compensatory Sit-to-St and Maneuvers in People With Dementia , ACSID ) . Methods : The ACSID covers the recall , initiation , and effective performance of compensatory STS movement maneuvers . The inter- and intrarater reliability , concurrent validity , sensitivity to change , and feasibility were investigated by secondary analysis of data of 97 participants from a r and omized controlled trial to improve motor-cognitive performances in people with mild to moderate dementia ( mean [ st and ard deviation ] age : 82.5 [ 5.9 ] years , Mini-Mental Status Examination : 21.9 [ 2.9 ] points ) . Concurrent validity of the individual ACSID items was assessed against reference criteria derived from video-motion analysis . Results : Good to excellent inter- ( kappa [ & kgr ; ] = 0.64 - 0.99 ; intraclass correlation coefficient [ ICC ] = 0.74 - 0.89 ) and intrarater ( & kgr;= 0.77 - 0.91 ; ICC = 0.77 - 0.91 ) , concurrent validity ( point-biserial correlation coefficients = |0.56|-|0.84| ) , and sensitivity to change ( st and ardized response means = 0.61 - 1.00 ) were found . Feasibility was excellent with a high completion rate ( 96.9 % ) , no critical events during assessment , and no floor or ceiling effects . Conclusions : The ACSID represents the first observation-based assessment instrument to document motor and cognitive aspects in the execution of a motor key feature in people with dementia , and has been shown to be reliable , valid , feasible , and sensitive to intervention-induced changes A prospect i ve cohort study was used to determine the reliability and validity of two fall risk assessment tools and  nurses ' clinical judgement in predicting patient falls .  The study wards comprised two aged care and rehabilitation wards within a 570 bed acute care tertiary teaching hospital in Western Australia . Instrument testing included test-retest reliability and calculations of sensitivity , specificity , positive predictive value , negative predictive value and accuracy . The test retest reliability of all methods was good . In this setting , the three methods of assessing fall risk showed good sensitivity but poor specificity . Also , all methods had limited accuracy , and overall , exhibited an inability to adequately discriminate between patient population s at risk of falling and those not at risk of falling . Consequently , neither nurses ' clinical judgement nor the fall risk assessment tools could be recommended for assessing fall risk in this clinical setting Objective To develop and vali date an instrument for use in the intensive care unit to accurately diagnose delirium in critically ill patients who are often nonverbal because of mechanical ventilation . Design Prospect i ve cohort study . Setting The adult medical and coronary intensive care units of a tertiary care , university-based medical center . Patients Thirty-eight patients admitted to the intensive care units . Measurements and Main Results We design ed and tested a modified version of the  Confusion Assessment Method  for use in intensive care unit patients and called it the CAM-ICU . Daily ratings from intensive care unit admission to hospital discharge by two study nurses and an intensivist who used the CAM-ICU were compared against the reference st and ard , a delirium expert who used delirium criteria from the Diagnostic and Statistical Manual of Mental Disorders ( fourth edition ) . A total of 293 daily , paired evaluations were completed , with reference st and ard diagnoses of delirium in 42 % and coma in 27 % of all observations . To include only interactive patient evaluations and avoid repeat-observer bias for patients studied on multiple days , we used only the first-alert or lethargic comparison evaluation in each patient . Thirty-three of 38 patients ( 87 % ) developed delirium during their intensive care unit stay , mean duration of 4.2 ± 1.7 days . Excluding evaluations of comatose patients because of lack of characteristic delirium features , the two critical care study nurses and intensivist demonstrated high interrater reliability for their  CAM-ICU ratings with kappa statistics of 0.84 , 0.79 , and 0.95 , respectively ( p < .001 ) . The two nurses ’ and intensivist ’s sensitivities when using the CAM-ICU compared with the reference st and ard were 95 % , 96 % , and 100 % , respectively , whereas their specificities were 93 % , 93 % , and 89 % , respectively . Conclusions The CAM-ICU demonstrated excellent reliability and validity when used by nurses and physicians to identify delirium in intensive care unit patients . The CAM-ICU may be a useful instrument for both clinical and research purpose s to monitor delirium in this challenging patient population OBJECTIVE The authors developed and evaluated the reliability and validity of the Brown Assessment of Beliefs Scale , a clinician-administered seven-item scale design ed to assess delusions across a wide range of psychiatric disorders . METHOD The authors developed the scale after review ing the literature on the assessment of delusions . Four raters administered the scale to 20 patients with obsessive-compulsive disorder ( OCD ) , 20 patients with body dysmorphic disorder , and 10 patients with mood disorder with psychotic features .  Audiotaped interviews of scale administration conducted by one rater were independently scored by the other raters to evaluate interrater reliability . The scale was administered to 27 patients twice to determine test-retest reliability . Other insight instruments as well as scales that assess symptom severity were administered to assess convergent and discriminant validity . Sensitivity to change was assessed in a multicenter treatment study of sertraline for OCD . RESULTS Interrater and test-retest reliability for the total score and individual item scores was excellent , with a high degree of internal consistency . One factor was obtained that accounted for 56 % of the variance . Scores on the Brown Assessment of Beliefs Scale were not correlated with symptom severity but were correlated with other measures of insight . The scale was sensitive to change in insight in OCD but was not identical to improvement in severity . CONCLUSIONS The Brown Assessment of Beliefs Scale is a reliable and valid instrument for assessing delusionality in a number of psychiatric disorders . This scale may help clarify whether delusional and nondelusional variants of disorders constitute the same disorder as well as whether delusionality affects treatment outcome and prognosis Objectives : The Eating Assessment Tool is a self-administered , symptom-specific outcome instrument for dysphagia . The purpose of this study was to assess the validity and reliability of the  10-item Eating Assessment Tool ( EAT-10 ) . Methods : The investigation consisted of 4 phases : 1 ) line-item generation , 2 ) line-item reduction and reliability , 3 ) normative data generation , and 4 ) validity analysis . All data were collected prospect ively . Internal consistency was assessed with the Cronbach alpha . Test-retest reliability was evaluated with the Pearson product moment correlation coefficient . Normative data were obtained by administering the instrument to a community cohort of healthy volunteers .  Validity was assessed by administering the instrument before and after dysphagia treatment and by evaluating survey differences between normal persons and those with known diagnoses . Results : A total of 629 surveys were administered to 482 patients . The internal consistency ( Cronbach alpha ) of the final instrument was 0.960 . The test-retest intra-item correlation coefficients ranged from 0.72 to 0.91 . The mean ( ±SD ) EAT-10 score of the normal cohort was 0.40 ± 1.01 . The mean EAT-10 score was 23.58 ± 13.18 for patients with esophageal dysphagia , 23.10 ± 12.22 for those with oropharyngeal dysphagia , 9.19 ± 12.60 for those with voice disorders , 22.42 ± 14.06 for those with head and neck cancer , and 11.71 ± 9.61 for those with reflux . The patients with oropharyngeal and esophageal dysphagia and a history of head and neck cancer had a significantly higher EAT-10 score than did those with reflux or voice disorders ( p < 0.001 ) . The mean EAT-10 score of the patients with dysphagia improved from 19.87 ± 10.5 to 5.2 ± 7.4 after treatment ( p < 0.001 ) . Conclusions : The EAT-10 has displayed excellent internal consistency , test-retest reproducibility , and criterion-based validity . The normative data suggest that an EAT-10 score of 3 or higher is abnormal . The instrument may be utilized to document the initial dysphagia severity and monitor the treatment response in persons with a wide array of swallowing disorders We translated the  Canadian residential long term care versions of the Alberta Context Tool ( ACT ) and the Conceptual Research Utilization ( CRU ) Scale into German , to study the association between organizational context factors and research utilization in German nursing homes . The rigorous translation process was based on best practice guidelines for tool translation , and we previously published methods and results of this process in two papers . Both instruments are self-report question naires used with care providers working in nursing homes . The aim of this study was to assess the factor structure , reliability , and measurement invariance ( MI ) between care provider groups responding to these instruments . In a stratified r and om sample of 38 nursing homes in one German region ( Metropolregion Rhein-Neckar ) , we collected question naires  from 273 care aides , 196 regulated nurses , 152 allied health providers , 6 quality improvement specialists , 129 clinical leaders , and 65 nursing students . The factor structure was assessed using confirmatory factor models . The first model included all 10 ACT concepts . We also decided a priori to run two separate models for the scale-based and the count-based ACT concepts as suggested by the instrument developers . The fourth model included the five CRU Scale items . Reliability scores were calculated based on the parameters of the best-fitting factor models . Multiple-group confirmatory factor models were used to assess MI between provider groups . Rather than the hypothesized ten-factor structure of the ACT , confirmatory factor models suggested 13 factors . The one-factor solution of the CRU Scale was confirmed . The reliability was acceptable ( > 0.7 in the entire sample and in all provider groups ) for 10 of 13 ACT concepts , and high ( 0.90–0.96 ) for the CRU Scale . We could demonstrate partial strong MI for both ACT models and partial strict MI for the CRU Scale . Our results suggest that the scores of the German ACT and the CRU Scale for nursing homes are acceptably reliable and valid . However , as the ACT lacked strict MI , observed variables ( or scale scores based on them ) can not be compared between provider groups . Rather , group comparisons should be based on latent variable models , which consider the different residual variances of each group OBJECTIVE To examine the psychometric properties of a newly developed clinician rating scale , the Liebowitz Social Anxiety Scale for Children and Adolescents ( LSAS-CA ) . METHOD A total of 154 children and adolescents participated in an assessment consisting of a diagnostic interview , the  LSAS-CA  , and other measures of psychopathology and impairment . Sixty-one of these children also participated in a second LSAS-CA administration , by a different rater blind to diagnosis , within 7 days of the initial assessment . RESULTS High internal consistency ( alpha = .90-.97 for full sample and .83-.95 for social phobia group ) and test-retest reliability ( intraclass correlation coefficient = 0.89 - 0.94 ) were obtained for LSAS-CA total and subscale scores .  LSAS-CA scores had stronger associations with measures of social anxiety and general impairment than with a measure of depression .  Subjects with social anxiety disorder had significantly higher  LSAS-CA scores than subjects with other anxiety disorders and healthy controls . A  LSAS-CA cutoff score of 22.5 represented the best balance of sensitivity and specificity when distinguishing between individuals with social phobia and normal controls , whereas a cutoff of 29.5 was optimal for distinguishing social phobia from other anxiety disorders . CONCLUSION Initial findings suggest that the LSAS-CA is a reliable and valid instrument for the assessment of social anxiety disorder
Only a limited number of the identified instruments for the assessment of impairments is both reliable and valid .
The objective of this study is to provide a critical overview of available instruments to assess impairments in patients with rheumatic disorders , and to recommend reliable and valid instruments for use in allied health care and rehabilitation medicine .