It's a common assumption that systematic reviews represent the pinnacle of evidence in the hierarchical pyramid. However, this only holds true when the systematic review includes Randomized Controlled Trials (RCTs). Indeed, a systematic review that includes homogenous RCTs represents the most robust form of evidence. When these RCTs are sufficiently similar, their data can be amalgamated into a single, large study using specific mathematical techniques, a process known as meta-analysis. In essence, a systematic review is typically understood to entail an analysis of a collection of RCTs, whereas meta-analysis denotes a specific method of scrutinizing those congruent RCTs.
However, not all systematic reviews fall under this category. For example, what about a systematic review (SR) that comprises cohort studies instead of RCTs? The term "systematic review" merely implies collecting data in a systematic manner. The Centers for Disease Control and Prevention (CDC) defines a systematic review as a comprehensive and critical collection and analysis of all evidence responding to a specific question, which must be defined clearly with specific inclusion and exclusion criteria. Thus, any data collected in a systematic manner can be termed a systematic review.(https://www.cdc.gov/library/researchguides/sytemsaticreviews.html) Therefore, one could write an SR based on collection of similar cohort studies.
This brings up another question: do an SR based on cohort studies and an SR based on RCTs carry the same weight in terms of evidence strength? The answer is a definite no. But, is a single RCT stronger evidence than an SR based on a cohort study? This is a challenging question to answer broadly. According to the Oxford (UK) CEBM Levels of Evidence, 'Systematic reviews (with homogeneity) of cohort studies' are weaker than 'All or none randomized controlled trials' but stronger than 'low-quality randomized controlled trials'.
Oxford (UK) CEBM Levels of Evidence
1a: | Systematic reviews (with homogeneity) of randomized controlled trials |
1b: | Individual randomized controlled trials (with narrow confidence interval) |
1c: | All or none randomized controlled trials |
2a: | Systematic reviews (with homogeneity) of cohort studies |
2b: | Individual cohort study or low quality randomized controlled trials (e.g. <80% follow-up) |
2c: | "Outcomes" Research; ecological studies |
3a: | Systematic review (with homogeneity) of case-control studies |
3b: | Individual case-control study |
4: | Case-series (and poor quality cohort and case-control studies) |
5: | Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles" |
Furthermore, many SRs are not based on "RCTs or cohort studies." Some are based on case reports, like the review on COVID-19-associated acute pancreatitis.
Rahimian Z, Feili A, Ghaderpanah R, et al. COVID-19-associated acute pancreatitis: a systematic review of case reports. Przegl Epidemiol. 2023;77(1):66-73. doi:10.32394/pe.77.07
Others are based on descriptive studies and don't provide any concrete evidence, such as the review on the impact of nursing leaders' behaviors on nursing staff performance. Take a look at its abstract.
Alsadaan N, Salameh B, Reshia FAAE, et al. Impact of Nurse Leaders Behaviors on Nursing Staff Performance: A Systematic Review of Literature. Inquiry. 2023;60:469580231178528. doi:10.1177/00469580231178528
Nursing leadership is critical in facilitating and improving nurse performance, which is essential for providing quality care and ensuring patient safety. The aim of this study is to explore the relationship between nursing leadership and nurse performance by understanding the leadership behaviors and factors that motivate nurses to perform well. To study the factors that nurses believe motivate them to perform better, a systematic review was undertaken, correlating these factors to leadership behaviors/styles. The PRISMA guidelines were followed to identify relevant articles. After applying the selection criteria, 11 articles were included in the final analysis. Overall, 51 elements that influence nurses' motivation to perform better were found and categorized into 6 categories, including autonomy, competencies, relatedness, individual nursing characteristics, relationships and support, and leadership styles/practices. It has been discovered that both direct and indirect nursing leadership behaviors affect nurses' performance. A better understanding of the factors that motivate nurses to perform well and facilitating them in the work environment through leadership behaviors/styles can improve nurses' performance. There is a need to increase research on nurse leadership and nurses' performance in the current innovative and technologically integrated work environment to identify new factors of influence.
There are also SRs based on other SRs or clinical guidelines. For example, reviews examining core recommendations for rheumatoid arthritis care or the role of artificial intelligence in head and neck cancer.
Conley B, Bunzli S, Bullen J, et al. What are the core recommendations for rheumatoid arthritis care? Systematic review of clinical practice guidelines [published online ahead of print, 2023 Jun 9]. Clin Rheumatol. 2023;10.1007/s10067-023-06654-0. doi:10.1007/s10067-023-06654-0
Mäkitie AA, Alabi RO, Ng SP, et al. Artificial Intelligence in Head and Neck Cancer: A Systematic Review of Systematic Reviews [published online ahead of print, 2023 Jun 8]. Adv Ther. 2023;10.1007/s12325-023-02527-9. doi:10.1007/s12325-023-02527-9
The key takeaway is that just because an article carries the title "Systematic Review," it doesn't automatically denote that the evidence presented within is robust or definitive.