Skip to Main Content

Systematic and Complex Reviews

Types of reviews

Systematic  Review

Definition

A systematic review is a protocol driven academic research paper using transparent, reproducible methodologies to look for answers to specific research questions

Goals

Answer a focused clinical question. Eliminates bias.

Question

Clearly defined & answerable clinical question.

Components

Registered protocol with pre-specified eligibility criteria.
Systematic & replicable search strategy.
Assessment of the validity of findings.
Interpretation and presentation of results.
Reference list.

No. of Authors

At least 3 for a formal systematic review to eliminate bias.

Timeline

6 to 12 months, average 12 months. 
See Timeline for a Cochrane review

Requirements

Thorough knowledge of topic.
Systematic searching of at least 3 relevant databases.
Statistical analysis resources (for meta-analysis) may be included.

Value

Connects practicing clinicians to high quality evidence.
Supports evidence based practice.

 

Systematic Review

Narrative Review

Definition

A systematic review is a protocol driven academic research paper using transparent, reproducible methodologies to look for answers to specific research questions

Qualitatively summarizes evidence on a topic using informal or subjective methods to collect and interpret studies. 

Goals

Answer a focused clinical question. Eliminates bias.

Provide a summary/overview of a topic.

Question

Clearly defined & answerable clinical question.

Can be a general topic or a specific question.

Components

Registered protocol with pre-specified eligibility criteria.
Systematic & replicable search strategy.
Assessment of the validity of findings.
Interpretation and presentation of results.
Reference list.

Introduction. 
Methods.
Discussion.
Conclusion.
Reference list.

No. of Authors

At least 3 for a formal systematic review to eliminate bias.

1 or more.

Timeline

6 to 12 months, average 12 months. 
See Timeline for a Cochrane review

Weeks to months.

Requirements

Thorough knowledge of topic.
Systematic searching of at least three relevant databases.
Statistical analysis resources (for meta-analysis) may be included.

Understanding of topic.
The number of databases searched is generally unknown. Some narrative reviews will perform and record a systematic search of selected databases.

Value

Connects practicing clinicians to high quality evidence.
Supports evidence based practice.

Provides a selective summary of literature on a topic.

References

Kysh, L. (2013). Difference between a systematic review and a literature review (Version 1). figshare. https://doi.org/10.6084/m9.figshare.766364

 

Systematic  Review

Scoping Review

Definition

A systematic review is a protocol driven academic research paper using transparent, reproducible methodologies to look for answers to specific research questions

A Scoping review is a protocol driven academic research paper using transparent methodologies to map or chart the literature related to a broader research question. The protocol may be developed iteratively.

Goals

Answer a focused clinical question. Eliminates bias.

Provide a summary/overview of a topic.

Question

Clearly defined & answerable clinical question.

Can be a general topic or a specific question.

Components

Registered protocol with pre-specified eligibility criteria.
Systematic & replicable search strategy.
Assessment of the validity of findings.
Interpretation and presentation of results.
Reference list.

Registered protocol with pre-specified eligibility criteria.
Comprehensive search strategy.

Synthesis of records located. May be charted.
Reference list.

No. of Authors

At least 3 for a formal systematic review to eliminate bias.

1 or more.

Timeline

6 to 12 months, average 12 months. 
See Timeline for a Cochrane review

6 to 12 months, average 12 months. 

Requirements

Thorough knowledge of topic.
Systematic searching of at least 3 relevant databases.
Statistical analysis resources (for meta-analysis) may be included.

Supplementary searching is recommended.

Understanding of topic.
Systematic searching of multiple databases.

Supplementary searching is recommended.

Value

Connects practicing clinicians to high quality evidence.
Supports evidence based practice.

Provides a summary of literature on a topic, can identify gaps in research, clarify concepts, identify the extent and characteristics of the research on a topic

References

Peters, M. D. J., Godfrey, C., McInerney, P., Munn, Z., Tricco, A. C., & Khalil, H. (2020). Chapter 11: Scoping reviews. In E. Aromataris & Z. Munn (Eds.). JBI manual for evidence synthesis. JBI. https://synthesismanual.jbi.global

 

Systematic  Review

Rapid Review

Definition

A systematic review is a protocol driven academic research paper using transparent, reproducible methodologies to look for answers to specific research questions

A rapid review is a protocol driven academic research paper using transparent, reproducible methodologies to look for answers to specific research questions. The scope of the review is limited or streamlined so the outcomes are achieved in a shorter time-frame.

Goals

Answer a focused clinical question. Eliminates bias.

Answer a focused clinical question. Eliminates bias.

Question

Clearly defined & answerable clinical question.

Clearly defined & answerable clinical question. The intervention considered or outcomes included may be limited. May include date and language restrictions.

Components

Registered protocol with pre-specified eligibility criteria.
Systematic & replicable search strategy.
Assessment of the validity of findings.
Interpretation and presentation of results.
Reference list.

Registered protocol with pre-specified eligibility criteria.
Systematic & replicable search strategy.
Assessment of the validity of findings.
Interpretation and presentation of results.
Reference list

No. of Authors

At least 3 for a formal systematic review to eliminate bias.

1-2 

Timeline

6 to 12 months, average 12 months. 
See Timeline for a Cochrane review

1 week-6 months

Requirements

Thorough knowledge of topic.
Systematic searching of at Medline, Embase and Cochrane databases as well as relevant subject databases and at least one multi-disciplinary database.
Statistical analysis resources (for meta-analysis) may be included.

Supplementary searching is recommended.

Thorough knowledge of topic.
Systematic searching of at Medline Embase and Cochrane relevant databases. Limit number or exclude subject databases.
Statistical analysis resources (for meta-analysis) may be included.

Limit or exclude supplementary searching 

Value

Connects practicing clinicians to high quality evidence.
Supports evidence based practice.

Provides "timely evidence for decision-making purposes including to address urgent and emergent health issues and questions deemed to be of high priority". 

References

Garritty, C., Gartlehner, G., Kamel, C., King, V. J., Nussbaumer-Streit, B., Stevens, A., Hamel, C., & Affengruber, L. (2020). Cochrane rapid reviews: Interim guidance from the Cochrane Rapid Reviews Methods Group. http://methods.cochrane.org/sites/methods.cochrane.org.rapidreviews/files/uploads/cochrane_rr_-_guidance-23mar2020-final.pdf

 

Systematic  Review

Umbrella Review

Definition

A systematic review is a protocol driven academic research paper using transparent, reproducible methodologies to look for answers to specific research questions

An Umbrella  review is a review of previously published systematic reviews or meta-analyses and represent one of the highest levels of evidence.

Goals

Answer a focused clinical question. Eliminates bias.

Provide a summary of available evidence related to a particular question.

Question

Clearly defined & answerable clinical question.

Clearly defined & answerable clinical question.

Components

Registered protocol with pre-specified eligibility criteria.
Systematic & replicable search strategy.
Assessment of the validity of findings.
Interpretation and presentation of results.
Reference list.

Registered protocol with pre-specified eligibility criteria.

Systematic & replicable search strategy.
Assessment of the validity of findings.
Interpretation and presentation of results.
Reference list.

 

No. of Authors

At least 3 for a formal systematic review to eliminate bias.

1 or more.

Timeline

6 to 12 months, average 12 months. 
See Timeline for a Cochrane review

Generally faster than systematic reviews.

Requirements

Thorough knowledge of topic.

Systematic searching of at least 3 relevant databases.
Statistical analysis resources (for meta-analysis) may be included.

Supplementary searching is recommended.

Thorough knowledge of topic.
Systematic searching of least 3 relevant databases as well as searches of major repositories of systematic reviews such as but not limited to Prospero, Cochrane Database of Systematic Reviews and JBI Evidence Synthesis. Also consider subject specific repositories such as OT Seeker or PEDro or EPPI Centre Evidence Library.

Supplementary searching is recommended.

Value

Connects practicing clinicians to high quality evidence.
Supports evidence based practice.

Highlight the evidence base around a particular topic, addressing what is consistent and what is not. 

References

Aromataris, E., Fernandez, R., Godfrey, C., Holly, C., Khalil, H., & Tungpunkom, P. (2020). Chapter 10: Umbrella reviews. In E. Aromataris & Z. Munn (Eds.). JBI manual for evidence synthesis. JBI. https://synthesismanual.jbi.global

Fusar-Poli, P., & Radua, J. (2018). Ten simple rules for conducting umbrella reviews. Evidence-Based Mental Health, 21, 95-100. http://doi.org/10.1136/ebmental-2018-300014

 

Systematic  Review

Integrative Review

Definition

A systematic review is a protocol driven academic research paper using transparent, reproducible methodologies to look for answers to specific research questions

"Critical analysis of empirical, methodological, or theoretical literature, which draws attention to future research needs" (Toronto & Remington, 2020).

Goals

Answer a focused clinical question. Eliminates bias.

To provide a comprehensive investigation of the subject and concepts.

Question

Clearly defined & answerable clinical question.

Review questions are broad and well defined.

Components

Registered protocol with pre-specified eligibility criteria.
Systematic & replicable search strategy.
Assessment of the validity of findings.
Interpretation and presentation of results.
Reference list.

Pre-specified eligibility criteria. 

Systematic & transparent  search strategy.
Quality appraisal.

Analysis and synthesis.

Generation of a conceptual level of knowledge.
Reference list.

No. of Authors

At least 3 for a formal systematic review to eliminate bias.

1 or more.

Timeline

6 to 12 months, average 12 months. 
See Timeline for a Cochrane review

6-12 months.

Requirements

Thorough knowledge of topic.
Systematic searching of at least 3 relevant databases.
Statistical analysis resources (for meta-analysis) may be included.

Supplementary searching is recommended.

Systematic searching  of multiple databases.

Supplementary searching is recommended.

Value

Connects practicing clinicians to high quality evidence.
Supports evidence based practice.

Provides an exploration and analysis of the current available research literature across different fields and perspectives, identifying directions for possible future research.

References

Cronin, M. A., & George, E. (2020). The why and how of the integrative Review. Organizational Research Methods. https://doi.org/10.1177/1094428120935507

Oermann, M. H., & Knafl, K. A. (2021). Strategies for completing a successful integrative review. Nurse Author & Editor, 31(3-4), 65-68. https://doi.org/10.1111/nae2.30  

Toronto, C. E., & Remington, R. (Eds.). (2020). A step-by-step guide to conducting an integrative review. Springer International Publishing. 

 

Systematic  Review

Realist Review

Definition

A systematic review is a protocol driven academic research paper using transparent, reproducible methodologies to look for answers to specific research questions

A realist review is a theory driven academic research paper using transparent methodologies to explore the how and why an intervention or program works and in which context. Causation is of particular interest, especially how that might be shaped by a social context (Wong, 2013).

Goals

Answer a focused clinical question. Eliminates bias.

To determine what works for whom under what circumstances, how and why, "to articulate underlying programme theories and then to interrogate the existing evidence to find out whether and where these theories are pertinent and productive" (Rycroft-Malone, 2012).

Question

Clearly defined & answerable clinical question.

Clearly defined question(s)

Components

Registered protocol with pre-specified eligibility criteria.
Systematic & replicable search strategy.
Assessment of the validity of findings.
Interpretation and presentation of results.
Reference list.

Planned protocol.

Iterative searches. Steps of the search recorded. Supplementary search strategies, especially citation tracking, backwards and forward have been shown to be very useful.

Appraisal of documents.

Analysis and synthesis with a focus on theory and theory building (Wong, 2013).

Interpretation and presentation of results.

Reference list.

No. of Authors

At least 3 for a formal systematic review to eliminate bias.

2 or more.

At least 2 for a formal systematic review to eliminate bias. Timeline

6 to 12 months, average 12 months. 
See Timeline for a Cochrane review

6 to 12 months, average 12 months.

Requirements

Thorough knowledge of topic.
Systematic searching of at least 3 relevant databases.
Statistical analysis resources (for meta-analysis) may be included.

Supplementary searching is recommended.

Expert knowledge of the area.

Systematic searching of at least 3 relevant databases.

Supplementary searching is essential.

Analysis and synthesis of the document located, identifying implications for policy and practice.

Value

Connects practicing clinicians to high quality evidence.
Supports evidence based practice.

"Attempts to provide policy makers with a transferable theory that suggests a certain program is more or less likely to work in certain respects, for particular subjects, in specific kinds of situations" (Saul, Willis & Best, 2013).

References

Rycroft-Malone, J., McCormack, B., Hutchinson, A. M., DeCorby, K., Bucknall, T. K., Kent, B., Schultz, A., Snelgrove-Clarke, E., Stetler, C. B., Titler, M., Wallin, L., & Wilson, V. (2012). Realist synthesis: Illustrating the method for implementation research. Implementation Science, 7(1), 33. https://doi.org/10.1186/1748-5908-7-33

Saul, J. E., Willis, C. D., Bitz, J., & Best, A. (2013). A time-responsive tool for informing policy making: Rapid realist review. Implementation Science, 8(1), 103. https://doi.org/10.1186/1748-5908-8-103 

Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R. (2013). RAMESES publication standards: Realist syntheses. BMC Medicine, 11(1), 21. https://doi.org/10.1186/1741-7015-11-21 

Wong, G., Westhorp, G., Manzano, A., Greenhalgh, J., Jagosh, J., & Greenhalgh, T. (2016). RAMESES II reporting standards for realist evaluations. BMC Medicine, 14(1), 96. https://doi.org/10.1186/s12916-016-0643-1

 

 

Systematic  Review

Qualitative Synthesis

Definition

A systematic review is a protocol driven academic research paper using transparent, reproducible methodologies to look for answers to specific research questions

A qualitative synthesis is a protocol driven academic research paper using transparent, reproducible methodologies to look for answers to specific research questions from a "respondents perspective" (Lachal, 2017).

Goals

Answer a focused clinical question. Eliminates bias.

Analyze and interpret responses in multiple papers to identify themes and connections and developing new conceptual knowledges.

Question

Clearly defined & answerable clinical question.

Broad and clearly defined question.

Components

Registered protocol with pre-specified eligibility criteria.
Systematic & replicable search strategy.
Assessment of the validity of findings.
Interpretation and presentation of results.
Reference list.

Registered protocol with pre-specified eligibility criteria.
Systematic & replicable search strategy.
Appraisal of resource quality.
Coding and Interpretation of results.
Reference list.

No. of Authors

At least 3 for a formal systematic review to eliminate bias.

2 or more.

Timeline

6 to 12 months, average 12 months. 
See Timeline for a Cochrane review

6 to 12 months, average 12 months. 

Requirements

Thorough knowledge of topic.
Systematic searching of at least 3 relevant databases.
Statistical analysis resources (for meta-analysis) may be included.

Supplementary searching is recommended.

Thorough knowledge of topic.
Systematic searching of at least 3 relevant databases.

Synthesis of resources.
Supplementary searching is recommended.

Value

Connects practicing clinicians to high quality evidence.
Supports evidence based practice.

Draws connections and comparisons between papers to create new knowledges and develop theory.

References

Lachal, J., Revah-Levy, A., Orri, M., & Moro, M. R. (2017). Metasynthesis: An original method to synthesize qualitative literature in psychiatry [Methods]. Frontiers in Psychiatry, 8. https://doi.org/10.3389/fpsyt.2017.00269

Boeije, H. R., van Wesel, F., & Alisic, E. (2011). Making a difference: Towards a method for weighing the evidence in a qualitative synthesis. Journal of Evaluation in Clinical Practice, 17(4), 657-663. https://doi.org/10.1111/j.1365-2753.2011.01674.x

Seers, K. (2012). What is a qualitative synthesis? Evidence Based Nursing, 15(4), 101. https://doi.org/10.1136/ebnurs-2012-100977

 

Systematic Review

Systematised Review

Definition

A systematic review is a protocol driven academic research paper using transparent, reproducible methodologies to look for answers to specific research questions

Includes elements of systematic review process while stopping short of systematic review. Typically conducted as postgraduate student assignment. (Grant & Booth, 2009)

Goals

Answer a focused clinical question. Eliminates bias.

Answer a well-defined research question

Question

Clearly defined & answerable clinical question.

Can be a general topic or a specific question.

Components

Registered protocol with pre-specified eligibility criteria.
Systematic & replicable search strategy.
Assessment of the validity of findings.
Interpretation and presentation of results.
Reference list.

Introduction. 

Methods

Results

Discussion

Strengths and limitations

Conclusion

Declaration of Interest

Reference List

No. of Authors

At least 3 for a formal systematic review to eliminate bias.

1 or more.

Timeline

6 to 12 months, average 12 months. 
See Timeline for a Cochrane review

Weeks to months.

Requirements

Thorough knowledge of topic.
Systematic searching of at least three relevant databases.
Statistical analysis resources (for meta-analysis) may be included.

What is known about the topic. May or may not include quality assessment. May or may not include comprehensive searching. 

Value

Connects practicing clinicians to high quality evidence.
Supports evidence based practice.

Provides a summary of literature on a topic.

 

Systematic Review

Mixed Methods Review

Definition

A systematic review is a protocol driven academic research paper using transparent, reproducible methodologies to look for answers to specific research questions

Any combination of methods where one significant component is a literature review (usually systematic). Within a review context it refers to a comprehensive synthesis of two or more types of data (qualitative or quantitative) and then aggregated into a final synthesis. (Grant & Booth, 2009)

Also referred to as Mixed methods synthesis, Mixed studies

Goals

Answer a focused clinical question. Eliminates bias.

The goal of a mixed methods review is to expand and strengthen a study’s conclusions

Question

Clearly defined & answerable clinical question.

Identify research gaps and determine the effects and appropriateness of interventions.  To answer multiple questions in one review

Components

Registered protocol with pre-specified eligibility criteria.
Systematic & replicable search strategy.
Assessment of the validity of findings.
Interpretation and presentation of results.
Reference list.

Descriptive title

Article’s ‘lead’ to attract the curiosity of readers

Research question/s

Essay summary

Implications

Findings

Methods

Bibliography

No. of Authors

At least 3 for a formal systematic review to eliminate bias.

1 or more.

Timeline

6 to 12 months, average 12 months. 
See 
Timeline for a Cochrane review

12 + months

Requirements

Thorough knowledge of topic.
Systematic searching of at least three relevant databases.
Statistical analysis resources (for meta-analysis) may be included.

Exhaustive searching is required.  The quality assurance for this type of review includes the use of a QA tool or checklist

 

Systematic Review

Critical Review

Definition

A systematic review is a protocol driven academic research paper using transparent, reproducible methodologies to look for answers to specific research questions

A critical review aims to demonstrate that the writer has extensively researched the literature and critically evaluated its quality. It goes beyond mere description of identified articles and includes a degree of analysis and conceptual innovation" and "an effective critical review presents, analyses and synthesizes material from diverse sources". "There is no formal requirement to present methods of the search, synthesis and analysis explicitly" (Grant & Booth 2009).

Goals

Answer a focused clinical question. Eliminates bias.

Demonstrate that the author has undertaken an exhaustive search to find all relevant articles within a research area and the quality of articles.

Question

Clearly defined & answerable clinical question.

Can be a general topic or a specific question.

Components

Registered protocol with pre-specified eligibility criteria.Systematic & replicable search strategy.
Assessment of the validity of findings.
Interpretation and presentation of results.
Reference list.

Introduction

Body

Conclusion

Questions to consider:

Objectives: What is the aim of the article

Theory: Theoretical framework or an important theoretical assumption

Concepts: Clearly define key concepts

Argument: What is the central argument or specific hypothesis

Method

Evidence: Is evidence provided?  How adequate is it?

Values: Clear or implicit value

Literature: How does your article add to the wider literature base?  Does it add any new knowledge?

Conclusion: A brief overall assessment

No. of Authors

At least 3 for a formal systematic review to eliminate bias.

1 or more.

Timeline

6 to 12 months, average 12 months. 
See 
Timeline for a Cochrane review

Weeks to months.

Requirements

Thorough knowledge of topic.
Systematic searching of at least three relevant databases.
Statistical analysis resources (for meta-analysis) may be included.

Requires an exhaustive search to find all the evidence 

Value

Connects practicing clinicians to high quality evidence.
Supports evidence based practice.

Evaluate the strengths and weaknesses of an article’s ideas and content.

 

Systematic Review

State-of-the-art Reviews

Definition

A systematic review is a protocol driven academic research paper using transparent, reproducible methodologies to look for answers to specific research questions

State-of-the-art reviews provide a time-based overview of the current state of knowledge within a research field and recommend direction for future research.  It will often highlight new ideas or gaps in the research.  No official quality assessment.

Goals

Answer a focused clinical question. Eliminates bias.

To summarise the current knowledge and provide priorities for future research

Question

Clearly defined & answerable clinical question.

Generate an interpretation of the literature informed by the expertise of the review team

Components

Registered protocol with pre-specified eligibility criteria.Systematic & replicable search strategy.
Assessment of the validity of findings.
Interpretation and presentation of results.
Reference list.

Providing a comprehensive discussion on a specific topic and summarise the current knowledge and advancements in that field

No. of Authors

At least 3 for a formal systematic review to eliminate bias.

1 or more.

Timeline

6 to 12 months, average 12 months. 
See 
Timeline for a Cochrane review

Weeks to months.

Requirements

Thorough knowledge of topic.
Systematic searching of at least three relevant databases.
Statistical analysis resources (for meta-analysis) may be included.

Requires a comprehensive search to find current evidence that addresses the purpose of the review

Value

Connects practicing clinicians to high-quality evidence.
Supports evidence-based practice.

Gathering current evidence in a specific research area and recommending direction for future research

 

Systematic Review

Review of Reviews|Overview

Definition

A systematic review is a protocol-driven academic research paper using transparent, reproducible methodologies to look for answers to specific research questions

Summary of the [medical] literature that attempts to survey the literature and describe its characteristics. (Grant & Booth, 2009)

Pollock M, Fernandes RM, Becker LA, Pieper D, Hartling L. Chapter V: Overviews of Reviews. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.4 (updated August 2023). Cochrane, 2023. Available from: https://training.cochrane.org/handbook/current/chapter-v

Goals

Answer a focused clinical question. Eliminates bias.

Collating evidence to demonstrate conclusive evidence for clinical decision-making.

Question

Clearly defined & answerable clinical question.

Evidence is used in clinical decision-making

Components

Registered protocol with pre-specified eligibility criteria.Systematic & replicable search strategy.
Assessment of the validity of findings.
Interpretation and presentation of results.
Reference list.

Introduction

Methods

Results

Discussion

Conclusion

References

No. of Authors

At least 3 for a formal systematic review to eliminate bias.

1 or more

Timeline

6 to 12 months, average 12 months. 
See 
Timeline for a Cochrane review

Weeks to months.

Requirements

Thorough knowledge of topic.
Systematic searching of at least three relevant databases.
Statistical analysis resources (for meta-analysis) may be included.

This type of review requires a comprehensive search of the literature in a given research area

Value

Connects practicing clinicians to high quality evidence.
Supports evidence based practice.

An Overview or Review of Reviews, collates multiple reviews to demonstrate conclusive evidence for clinical decision-making

See the below articles for a list of other types of reviews.

Barry, E.S., Merkebu, J. & Varpio, L. (2022). State-of-the-art literature review methodology: A six-step approach for knowledge synthesis. Perspect Med Educ 11, 281–288. https://doi.org/10.1007/s40037-022-00725-9 

Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26(2), 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family: Exploring review types and associated information retrieval requirements. Health Information & Libraries Journal, 36(3), 202-222. https://doi.org/10.1111/hir.12276

 

© Western Sydney University, unless otherwise attributed.
Library guide created by Western Sydney University Library staff is licenced under a Creative Commons Attribution 4.0 International (CC BY)