data, it is considered more reliable as it provides better evidence, as the more data included in reviews, the more confident we can be of conclusions. when appropriate, some systematic reviews include a meta-analysis, which uses statistical methods to combine data from multiple sources. a review might use quantitative data, or might employ a qualitative meta-synthesis, which synthesises data from qualitative studies. a review may also bring together the findings from quantitative and qualitative studies in a mixed methods or overarching synthesis. the combination of data from a meta-analysis can sometimes be visualised. one method uses a forest plot (also called a blobbogram). in an intervention effect review, the diamond in the 'forest plot' represents the combined results of all the data included. an example of a 'forest plot' is the cochrane collaboration logo. the logo is a forest plot of one of the first reviews which showed that corticosteroids given to women who are about to give birth prematurely can save the life of the newborn child. recent visualisation innovations include the albatross plot, which plots p-values against sample sizes, with approximate effect-size contours superimposed to facilitate analysis. the contours can be used to infer effect sizes from studies that have been analysed and reported in diverse ways. such visualisations may have advantages over other types when reviewing complex interventions. assessing the quality (or certainty) of evidence is an important part of some reviews. grade (grading of recommendations, assessment, development and evaluations) is a transparent framework for developing and presenting summaries of evidence and is used to grade the quality of evidence. the grade-cerqual (confidence in the evidence from reviews of qualitative research) is used to provide a transparent method for assessing the confidence of evidence from reviews or qualitative research. communication and dissemination once these stages are complete, the review may be published, disseminated and translated into practice after being adopted as evidence. the uk national institute for health research (nihr) defines dissemination as ‘getting the findings of research to the people who can make use of them to maximise the benefit of the research without delay’. however, many evidence users do not have time to invest in reading large and complex documents and/or may lack awareness or be unable to access newly published research. researchers are therefore developing skills to use creative communication methods such as illustrations, blogs, infographics and board games to share the findings of systematic reviews. automation of systematic reviews living systematic reviews are a relatively new kind of high quality, semi-automated, up-to-date online summaries of research which are updated as new research becomes available. the essential difference between a living systematic review and a conventional systematic review is the publication format. living systematic reviews are 'dynamic, persistent, online-only evidence summaries, which are updated rapidly and frequently'. while living systematic reviews seek to maintain current evidence, the automation or semi-automation of the systematic process itself is increasingly being explored. while little evidence exists to demonstrate it is as accurate or involves less manual effort, efforts that promote training and using artificial intelligence for the process are increasing. research fields history of systematic reviews in medicine a 1904 british medical journal paper by karl pearson collated data from several studies in the uk, india and south africa of typhoid inoculation. he used a meta-analytic approach to aggregate the outcomes of multiple clinical studies. in 1972 archie cochrane wrote: 'it is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials'. critical appraisal and synthesis of research findings in a systematic way emerged in 1975 under the term 'meta analysis'. early syntheses were conducted in broad areas of public policy and social interventions, with systematic research synthesis applied to medicine and health. inspired by his own personal experiences as a senior medical officer in prisoner of war camps, archie cochrane worked to improve how the scientific method was used in medical evidence, writing in 1971: 'the general scientific problem with which we are primarily concerned is that of testing a hypothesis that a certain treatment alters the natural history of a disease for the better'. his call for the increased use of randomised controlled trials and systematic reviews led to the creation of the cochrane collaboration, which was founded in 1993 and named after him, building on the work by iain chalmers and colleagues in the area of pregnancy and childbirth. current use of systematic reviews in medicine many organisations around the world use systematic reviews, with the methodology depending on the guidelines being followed. organisations which use systematic reviews in medicine and human health include the national institute for health and care excellence (nice, uk), the agency for healthcare research and quality (ahrq, usa) and the world health organization. most notable among international organisations is cochrane, a group of over 37,000 specialists in healthcare who systematically review randomised trials of the effects of prevention, treatments and rehabilitation as well as health systems interventions. when appropriate, they also include the results of other types of research. cochrane reviews are published in the cochrane database of systematic reviews section of the cochrane library. the 2015 impact factor for the cochrane database of systematic reviews was 6.103, and it was ranked 12th in the medicine, general & internal category. there are several types of cochrane review, including: intervention reviews assess the benefits and harms of interventions used in healthcare and health policy. diagnostic test accuracy reviews assess how well a diagnostic test performs in diagnosing and detecting a particular disease. for conducting diagnostic test accuracy reviews, free software such as metadta and cast-hsroc in the graphical user interface is available. methodology reviews address issues relevant to how systematic reviews and clinical trials are conducted and reported. qualitative reviews synthesize qualitative evidence to address questions on aspects other than effectiveness. prognosis reviews address the probable course or future outcome(s) of people with a health problem. overviews of systematic reviews (oors) are a new type of study to compile multiple evidence from systematic reviews into a single document that is accessible and useful to serve as a friendly front end for the cochrane collaboration with regard to healthcare decision-making. these are sometimes referred to as 'umbrella reviews'. living systematic reviews are continually updated, incorporating relevant new evidence as it becomes available. they are a relatively new kind of review, with methods still being developed and evaluated. they can be high quality, semi-automated, up-to-date online summaries of research which are updated as new research becomes available. the essential difference between a 'living systematic review' and a conventional systematic review is the publication format. living systematic reviews are 'dynamic, persistent, online-only evidence summaries, which are updated rapidly and frequently'. rapid reviews are a form of knowledge synthesis that 'accelerates the process of conducting a traditional systematic review through streamlining or omitting specific methods to produce evidence for stakeholders in a resource-efficient manner'. reviews of complex health interventions in complex systems review interventions and interventions delivered in complex systems to improve evidence synthesis and guideline development at a global, national or health systems level. the cochrane collaboration provides a handbook for systematic reviewers of interventions which 'provides guidance to authors for the preparation of cochrane intervention reviews.' the cochrane handbook also outlines the key steps for preparing a systematic review and forms the basis of two sets of standards for the conduct and reporting of cochrane intervention reviews (mecir - methodological expectations of cochrane intervention reviews). it also contains guidance on how to undertake qualitative evidence synthesis, economic reviews and integrating patient-reported outcomes into reviews. the cochrane library is a collection of databases that contains different types of independent evidence to inform healthcare decision-making. it contains a database of systematic review and meta-analyses which summarize and interpret the results of multi-disciplinary research. the library contains the cochrane database of systematic reviews (cdsr), which is a journal and database for systematic reviews in health care. the cochrane library also contains the cochrane central register of controlled trials (central) which is a database of reports of randomized and quasi-randomized controlled trials. the cochrane library is also available in spanish. the cochrane library is owned by cochrane. it was originally published by update software and now published by the share-holder owned publisher john wiley & sons, ltd. as part of wiley online library. royalties from sales of the cochrane library are the major source of funds for cochrane (over £6 million in 2017). there are 3.66 billion people around the world who have access to the library through national licences (national licences provide access to 1.5 billion individuals) or free provision for populations in low- and middle-income countries eligible under the who's hinari initiative. authors must pay an additional fee for their review to be truly open access. cochrane has an annual income of $10m usd. public involvement and citizen science in systematic reviews cochrane has several tasks that the public or other 'stakeholders' can be involved in doing, associated with producing systematic reviews and other outputs. tasks can be organised as 'entry level' or higher. tasks include: joining a collaborative volunteer effort to help categorise and summarise healthcare evidence data extraction and risk of bias assessment translation of reviews into other languages a recent systematic review of how people were involved in systematic reviews aimed to document the evidence-base relating to stakeholder involvement in systematic reviews and to use this evidence to describe how stakeholders have been involved in systematic reviews. thirty percent involved patients and/or carers. the active framework provides a way to consistently describe how people are involved in systematic review, and may be used as a way to support the decision-making of systematic review authors in planning how to involve people in future reviews. standardised data on initiatives (stardit) is another proposed way of reporting who has been involved in which tasks during research, including systematic reviews. while there has been some criticism of how cochrane prioritises systematic reviews, a recent project involved people in helping identify research priorities to inform future cochrane reviews. in 2014, the cochrane-wikipedia partnership was formalised. this supports the inclusion of relevant evidence within all wikipedia medical articles, as well as other processes to help ensure that medical information included in wikipedia is of the highest quality and accuracy. learning resources cochrane has produced many learning resources to help people understand what systematic reviews are, and how to do them. most of the learning resources can be found at the 'cochrane training' webpage, which also includes a link to the book testing treatments, which has been translated into many languages. in addition, cochrane has created a short video what are systematic reviews which explains in plain english how they work and what they are used for. the video has been translated into multiple languages, and viewed over 192,282 times (as of august 2020). in addition, an animated storyboard version was produced and all the video resources were released in multiple versions under creative commons for others to use and adapt. the critical appraisal skills programme (casp) provides free learning resources to support people to appraise research critically, including a checklist which contains 10 questions to 'help you make sense of a systematic review'. social, behavioural and educational in 1959, social scientist and social work educator barbara wootton published one of the first contemporary systematic reviews of literature on anti-social behavior as part of her work, social science and social pathology. several organisations use systematic reviews in social, behavioural, and educational areas of evidence-based policy, including the national institute for health and care excellence (nice, uk), social care institute for excellence (scie, uk), the agency for healthcare research and quality (ahrq, usa), the world health organization, the international initiative for impact evaluation (3ie), the joanna briggs institute and the campbell collaboration. the quasi-standard for systematic review in the social sciences is based on the procedures proposed by the campbell collaboration, which is one of several groups promoting evidence-based policy in the social sciences. the campbell collaboration: 'helps people make well-informed decisions by preparing, maintaining and disseminating systematic reviews in education, crime and justice, social welfare and international development.' the campbell collaboration is a sibling initiative of cochrane, and was created in 2000 at the inaugural meeting in philadelphia, usa,