1 Systematic Review and Meta-analysis

1.1 What is Systematic Review?

Systematic reviews are literature reviews that identify, obtain, filter, evaluate, and synthesize scientific studies to answer a research question. These reviews use transparent methods and, as far as possible, are based on technically reasoned and impartial decisions, and are therefore reproducible.

Suggested references: Cook et al. (1997);Gopalakrishnan and Ganeshkumar (2013);Delgado-Rodríguez and Sillero-Arenas (2018).

*Nota: uma revisão sistemática pode ou não ser seguida de uma meta-análise.

1.2 What is Meta-analysis?

Meta-analysis is a statistical method that combines the quantitative results of two or more independent primary studies, synthesizing them into a single result.

Suggested references:Haidich (2010),Vesterinen et al. (2014),Møller and Myles (2016), andWormald and Evans (2018).

1.3 What is Systematic Review and Living Meta-Analysis?

Systematic reviews and meta-analyses can quickly become obsolete. A living systematic review and living meta-analysis is a systematic review which is continually updated, incorporating new relevant evidence as it becomes available (fromLiving Systematic Review).

Suggested references:Elliott et al. (2014),Garner et al. (2016),Elliott et al. (2017), andSimmonds et al. (2022).

1.4 Systematic Reviews and Meta-Analyses in Pharmacology

In the context of pharmacology, individual studies are not always sufficient to provide reliable information for or against the use of a drug in an experiment, a project, or even in the development of a drug.

Thus, with or without meta-analysis, systematic reviews are helpful in gathering, combining, and synthesizing evidence relevant to the decision-making process in a single study.

Although abundant, the pharmacological literature is heterogeneous, and primary studies generally have small samples. Therefore, meta-analyses can be helpful in pharmacology. In addition, meta-analyses can help identify the beneficial or toxic effects of drugs unnoticed in primary studies.

Systematic reviews, with or without meta-analysis, can also be used to understand the mechanisms of action of a drug or a pharmacological class, identify gaps in knowledge, or study the effectiveness and safety of a drug or a pharmacological class.

Suggested references:Lino de Oliveira et al. (2020);Hesen et al. (2017);Al-Waeli et al. (2021). See systematic reviews in pharmacology in the"Portfolio & Initiatives"

2 General plan of systematic reviews and meta-analyses

2.1 Planning, implementation, publication, and maintenance

At SysPhar, the systematic reviews and meta-analysis is divided into chronologically organized phases as shown in Figure 5. Phase 0 is the period before starting. Phases 1 to 5 are for elaborating the research question, elaborating the protocol, registering the protocol, implementing the protocol, and publishing the results. Phase 6 is maintaining the systematic review and living meta-analysis.

*NOTE: Not every systematic review needs to be followed by a meta-analysis. Not every systematic review and meta-analysis needs to be conducted. Every pharmacologist must decide according to their own research area.

3 Phase 0: Before starting

3.1 Relevance of the study

To save time and effort, we suggest that pharmacologists conduct a preliminary analysis of the context of a systematic review and meta-analysis in pharmacology by answering the following questions:

If you answered "yes" to questions 1–5 and "no" to question 6, it is possible that your review has scientific, technological, and originality relevance, so prepare your team!

If you have also answered "yes" to question 6, that is, a review of your interest is already in progress, consider contacting the review team for possible collaboration.

*NOTE: To verify that the topic has been previously revised, a brief search on virtual bases relevant to Pharmacology (e.g., Medline viaPubMed
, Scopus,
Web of Science
, Embase) can be conducted. In addition, consider prepress platforms such asbioRxiv,medRxiv, OSF, or ongoing systematic review deposits such asPROSPERO.

In the planning, implementation, publication, and maintenance of a systematic review and meta-analysis in pharmacology, an ideal team would be formed by:

3.2 Preparing the team to begin

• Pharmacologists: selection of the theme and preparation of the question of the relevant review in pharmacology; preparation of the protocol (e.g., helping to identify the relevant aspects for the scope of the search and selection of studies, as well as a meta-analysis of the data).
• Librarians or methodologists specialized in systematic reviews of the literature: preparation of the protocol (e.g., helping to create search strategies in the literature); recommendation of software, tools, and approaches to search and select relevant literature to answer the review question.
• Statisticians or methodologists specialized in meta-analysis: preparation of the protocol (e.g., helping to select the types of calculations of effect size and statistical model according to the outcomes of interest), recommendation of software, tools, and approaches to analyze the data.
• Reviewers: realization of the less-specialized stages of the process (e.g., selection of studies applying pre-established criteria, extraction of data according to the protocol, and tabulation of data extracted from the studies). Even inexperienced reviewers can contribute to the implementation steps of the review protocol under the supervision of more experienced team members. These reviewers can be recruited during this work.

It can be helpful to create specific roles for each team member. More than one person can play each role, and each person can play more than one role simultaneously:

• Leader:responsible for managing team activities and reviewing documents**.
• Independent reviewer 1:responsible for applying study eligibility criteria, extracting, and tabulating data from studies included in the review independently of reviewer 2
• Independent reviewer 2:responsible for applying the eligibility criteria of the studies, extracting, and tabulating the data of the studies included in the review independently of reviewer 1
• Reviewer 3 (the conciliator):responsible for reconciling the discrepancies between independent reviewers 1 and 2.

*NOTE: At the beginning of the review process, it is essential to record and store all review-related documents in one sharing location (e.g.,OSF, Google Drive,OneDrive) for the entire team to have access.

4 Phase 1: Drafting the review question

The definition of the research question will guide all decisions in the next phases of a systematic review and meta-analysis. Often, research questions in pharmacology assume similar structures:

• "Is drug X effective in curing/improving a Y condition in a specific type of experiment/test?";
• "Is there a relationship between dose-response between drug X and answer Y?";
• "Is a population (such as rats, mice, humans, and in vitro preparation) susceptible to the effects of drug X?";
• “Is a subpopulation of the sample (such as species, lineage, and sex) more susceptible to the effects of drug X?".

Thus, mnemonic tools were developed to aid in elaborating a clear, objective research question that meets the interests of the review.

4.1 PICO Tool

The PICOtool (P: patient or population or problem; I: intervention; C: comparison or control; O: outcome), developed to review randomized controlled clinical trials, is also suitable for reviewing nonclinical studies in pharmacology.

NOTE: Note that in the context of Pharmacology, the term "Intervention" is often synonymous with "treatment with drug X."

Suggested references:Eriksen and Frandsen (2018)andSchardt et al. (2007).

4.2 Other mnemonic tools

Other mnemonic tools can be used in pharmacology and are helpful in asking questions about specific types of research, such as in vitro, in vivo, and ex vivo.

Examples of variations to the PICO tool, PICOC (PICO plus Contexto), PICOT (PICO plus Time), SPICE (S: Setting or experimental context; P: Population; I: Intervention; C: Comparison; E: Evaluation or Evaluation); and SPIDER (S - Sample or Sample of interest; P - Phenomenon or Phenomenon of Interest; D - Experimental Design; E - Evaluation or Evaluation; R - Research type).

Suggested reference:Cooke et al. (2012)and Wohlin et al. (2012).

5 Phase 2: Drafting the revision protocol

5.1 Protocol templates

The protocol contains a description of all the methods selected to answer the research question prepared in the first phase of the process.

Specialized organizations offer suggestions for protocols following good practices for systematic review and meta-analysis of studies in general ( PRISMA), humans (COCHRANE), and animals (CAMARADES
,SYRCLE).

We suggest using the structured Form SYRCLE for the elaboration of a complete protocol. The form in .docx can be downloaded HERE.

See examples of complete protocols of systematic reviews and meta-analyses in pharmacology (Zameer et al., 2019; Abdel et al., 2021; Ma et al., 2018; Nibber et al., 2020).

5.2. Planning the search strategy

The protocol should have a detailed plan for obtaining relevant publications from the bibliographic databases called "search strategy." *

A search strategy consists of describing the lists of relevant search terms (keywords) combined with Boolean operators (e.g., AND, OR) and syntaxes used in searches conducted in different bibliographic databases.

See search strategy examples using terms related to population (P), intervention (I), and outcome (O) of interest or their combination.

Examples of protocols presenting search strategies in pharmacology (Zameer et al., 2019; Abdel et al., 2021; Ma et al., 2018; Nibber et al., 2020).

*NOTE:

1-Small searches in the bibliographic databases, called pilot studies, help to establish the final search strategy presented in the protocol;

2- research on different bibliographic bases combined will provide a more comprehensive review;

3- different search engines operate differently;

4- access to some bibliographic bases requires payment or institutional login.

5.3. Planning the selection strategy

The authors describe how relevant studies will be selected from publications retrieved from the bibliographic databases in this protocol item.

The selection strategy includes planning screening phases (title and/or summary and/or full text), the number of independent reviewers, and the conciliators involved. Other decisions include contacting the corresponding authors when there is a lack of information.

The included studies are the most relevant publications to answer the review questions and will be analyzed in a systematic review and meta-analysis.

Examples of protocols include selection strategies in the field of pharmacology (Ballard et al., 2019; Zameer et al., 2019; Pettorruso et al., 2019; David et al., 2019; Sartim et al., 2020).

5.4. Planning data extraction

In this section, the authors describe the approaches to extract qualitative and quantitative data from the studies included in the review. Ideally, this step should be performed in duplicate, with two independent reviewers, to avoid errors. The protocol should also analyze agreements between reviewers (e.g.,Cohen Kappa).

Qualitative data are extracted to evaluate the external (degree of generalization of results) and internal (risk of experimental bias) studies included in the review.

To evaluate the external validity, information related to the characteristics of the studies defined during the preparation of the question using the mnemonic tool is usually extracted. For example, if the mnemonic tool applied was the PICO, you may want to extract and tabulate information about the population studied (e.g., species, age, and gender), pharmacological intervention (e.g., type of compound, drug class, dose, route of administration, duration of treatment), comparison or control (e.g., type of compound, route of administration, duration of treatment), and outcomes (e.g., method of obtaining, type of metric, unit of measurement, primary analysis).

To evaluate internal validity, the information required by specific tools for assessing the risk of bias is generally extracted from each primary study. In the area of pharmacology, we suggest the use of the following tools:

§ RoB-SYRCLE adapted for evaluation of studies with animal experimentation from the Cochrane. At the Tools & Resources linkwe provide a quick guide to its use.

§ AMSTAR 2, the RoB tool, applies to evaluating the internal quality of systematic reviews, including randomized or non-randomized studies of health interventions.

Numerical data (e.g., mean, standard deviation, sample sizes, number of comparisons, p, F or t values, correlation indices or any other measure, manipulation, or transformation of the data) are necessary for the calculation of effect sizes of each primary study and for conducting the meta-analysis.

Examples include internal quality planning assessment (Ma et al., 2018; Souza et al., 2019; Carabali et al., 2021; Williamson et al., 2016; Azab et al., 2020), and the evaluation of the external quality of studies in pharmacology (Lino de Oliveira et al., 2020; Azab et al., 2020; Sartim et al., 2020).

5.5. Planning Meta-Analysis

In this protocol, the authors describe the plans to calculate the effect size of each study and meta-analysis.

5.5.1 Effect size calculations

Protocols should explain how effect sizes will be calculated (e.g., mean difference, odds ratio) using numerical data extracted from primary studies. In addition, indicators of uncertainty in the size of the effect sizes should be mentioned. In general, the effect sizes are presented with confidence intervals.

Effect sizes were used to estimate the difference between outcome values (i.e., outcome or dependent variable) in experimental groups (e.g., control group versus treated group) or the strength of the relationship between an outcome (i.e., outcome or dependent variable) and intervention (e.g., treatment with a drug).

In addition to the type of relationship between variables that one wants to estimate, the type of outcome of interest (continuous? dichotomous?) will also influence the choice of effect size calculation. In addition, the decision on how the estimated effect sizes in primary studies will be combined and the indicators of the uncertainty of the joint effect size (confidence intervals, standard error, standard deviation, and heterogeneity).

Table 2.Examples of calculations of effect sizes in differences between independent group means

Notes: Adapted table ofEspírito-Santo and Daniel (2015). M = mean of each group; SD = standard deviation of each group; n = number of subjects; gl = degrees of freedom (n -1); control = control. The table is based on the following references:Cohen (1992);Borenstein (2009);Cummings (2012);Hedges (1981);Lakens (2013)andRosenthal (1994).

In pharmacology, the size of the effect could be interpreted as the effect of treatment with the drug.

In a hypothetical experiment, pharmacologists could assess the hypothesis that the influence of drug X on a given Y characteristic of a sample of P interest is greater than the influence of control C ("compared to control C, what effect is drug X on measure Y taken in population P?").

In this case, the calculation of the difference between the Y means obtained in the P sample treated with C or X would indicate the size of the difference between the treatments.

Examples of protocols for planning effect sizes of studies in the field of pharmacology (Lino de Oliveira et al., 2020; Carabali et al., 2021; Sartim et al., 2020).

5.5.2 Meta-Analysis

At this stage, the authors should specify, by the outcome, whether a meta-analysis will be conducted, and how this decision will be made. In addition, the protocol should have a description of how the studies will be grouped to calculate the size of the combined effect, which statistical model is appropriate for combining the effect sizes of the primary studies, which indicators of uncertainty will be calculated, and how publication bias will be investigated:

§ Meta-analysis: yes or no? The number of studies available for calculation, which are like each other, is a limiting factor for meta-analysis. Theoretically, two studies were sufficient to calculate the joint effect size. However, as with other statistical methods, a meta-analysis can provide inconclusive results with small sample sizes. Therefore, although sample size calculations or statistical power analysis can be performed, the protocol should have a description of these calculations.

§ How will the studies be grouped to calculate the size of the joint effect? According to the research area, it is up to the authors to define how similar the studies should be to make up a group or subgroup of studies that will have the joint results. Examples: a meta-analysis will be performed with all studies with the same type of population (or intervention); stratified meta-analysis will be performed with studies divided into subgroups of species/strain/sex/age of experimental animals, types and doses of drugs, or method of obtaining the outcome.

§ Statistical model of meta-analysis: Fixed or random models? Even when investigating the same hypothesis, the primary studies in pharmacology vary in experimental design and ways of doing it. Because of this heterogeneity, random effect models are often eligible for meta-analysis in research fields such as pharmacology.

§ What uncertainty indicators of the joint effect size estimate? In general, each joint effect size estimated in a stratified meta-analysis or meta-analysis is accompanied by confidence intervals, standard error, standard deviation, and heterogeneity (e.g., statistics Q, I²).

§ How will publication bias be investigated? Publication bias, when planned, can be evaluated using the funnel and trim-and-fillplotting methods.

Examples of protocols describing plans for meta-analysis in the field of pharmacology (Yamato et al., 2014; Soliman et al., 2019; Lino de Oliveira et al., 2020; Carabali et al., 2021; Sartim et al., 2020).

Suggested references: Cochrane Handbook,Sterne et al. (2011),Vesterinen et al. (2014),Borenstein et al. (2009), andShi and Lin (2019).

5.6. Planning the publication of results

Systematic reviews often involve different employees and large groups of reviewers. It is up to the authors of the protocol to specify, even vaguely, how all taxpayers will receive credit for their work to accommodate the expectations of all authors involved. It is up to the authors to find the means of communication to make the results available to the competent public.

Examples of protocols describe dissemination plans in the field of pharmacology (Kavanagh et al., 2019; Lee et al., 2019).

5.7. Planning the living systematic review

Finally, pharmacologists may choose whether to keep their systematic reviews and living meta-analyses.

TheCochrane Collaborationguides the design methods, production, and publication of living systematic reviews in theCochrane Database of Systematic Reviews.

CAMARADES BR offers a form proposal, adapted from the Cochrane collaboration, for systematic reviews and living meta-analyses in Pharmacology. Download: HERE.

6 Phase 3: Registering the review protocol

6.1 Public registration platforms

Registering the protocol before implementing systematic reviews and meta-analyses can help reviewers adhere to the plan. An extensive list of decisions must be made regarding the process, reducing the incidence of biased reviews. It may be necessary to return to the protocol at any stage of the review to find the information necessary to implement the review activities.

Pharmacologists can record their reviews on public platforms, such as PROSPERO and the Open Science Framework (OSF):

§ PROSPERO: a platform specialized in protocol records for systematic reviews and meta-analyses of studies relevant to human health in humans and laboratory animals

§ Open Science Framework: free public platform used to deposit scientific protocols of any study, including systematic reviews and meta-analysis protocols.

NOTE: Records can also be made in peer-review scientific journals, see examples:Soliman et al., 2019; Lee et al., 2019; Kavanagh et al., 2019; Ramos-Hryb et al., 2019; Ang et al., 2020; Maguire and Guérin, 2020, Thombs et al., 2020 , Bolzan e Lino de Oliveira, 2021.

7 Phase 4: Implementation of the review protocol

The review protocol in phase 4 consists of implementing the actions foreseen in the protocol previously elaborated in phase 2 and registered in phase 3).

Throughout the implementation process, reviewers need to make complete notes of any decisions made and any deviations from the protocol.

These annotations increase transparency and help ensure that all team members are in line with expectations. In addition, they help review readers judge the degree of trust in the data.

7.1. Free software, scripts, and other features

We identified many free or commercial resources available to implement the processes in a systematic review and meta-analysis that can help pharmacologists. The degrees of automation of these tools vary, but they generally accelerate the activities needed to perform the review.

7.1.1 Search on bibliographic bases.

The "advanced search" menu of virtual bibliographic database search engines (e.g., Medline via Pubmed, Scopus,Web of Science, Embase) is often more appropriate for the type of search required in a Systematic Review than the simple search menu.

Retrieving documents from different virtual bases requires the use of reference management software for deduplication.Mendeley, Zotero, and Rayyan are examples of free managers. CAMARADES developed an automated systematic search duplication tool (ASySD).

The number of publications obtained in the research conducted in each bibliographic database, before and after deduplication, should be noted in the PRISMA flowchart presented in the final report of the review. The PRISMA flowchart can be downloaded HERE.

7.1.2 Screening of relevant studies

Reference management software is also helpful for applying the selection and eligibility criteria of the relevant studies. For example, Mendeleyand Rayyan can semi-automatically sort references in a library.

Other free resources are ASReview, Parsifal, Rvtools (tools for r-evidence synthesis),Sysrev, Screenatron, and Systematic Review Accelerator, and SyRF (CAMARADES).

The results of the screening processes (number of excluded studies, exclusion reasons, number of studies included) should be noted in a PRISMA flowchart presented in the final report of the review. The PRISMA flowchart can be downloaded HERE.

7.1.3 Data extraction

Reference managers (Mendeley, Zotero, and Rayyan) can obtain bibliographic information from the included studies (e.g., author name, year, journal). Online tools, such as Colandr and SyRF, help annotate, tabulate, share, and manage qualitative and numerical data extracted from publications more efficiently.

There are free versions of tools such as Engauge Digitizer or WebPlotDigitizer that convert images (from charts or maps) into numbers. In addition, manual measurements of graphics elements (scale or bar sizes or line extensions and other attributes) are possible using the digital ruler of PDF readers (for example, free version of Adobe reader).

Some programming language knowledge allows users to use the metaDigitise Rand metagear package to extract descriptive statistics. Tables in CSV format can be formatted for use in future meta-analyses (Table 1).

The pacman package is an R-pack management tool, which combines dplyr, irr, and rel functionality, used for matching calculations between reviewers (Cohen Kappa) whenever two independent reviewers perform an activity. We provide the table models here at SysPhar .csv for analysis of agreement between reviewers (Table 2) and scripts for the Pacman tool.

7.1.4 Bias risk assessment using the RoB-Syrcle tool

The presentation of the data from the RoB-Syrcle tool can be facilitated by the Robvis tool (McGuinness and Higgins, 2020). Here, we provide the SysPhar .csv table models for risk assessment of bias (Table 3) and scripts for the use of the Robvis tool.

Legend: The column on the left corresponds to the signaling questions. The legend corresponds to the judgments "low risk of bias" (green), "risk of uncertain bias" (yellow) and "high risk of bias" (red). Simulation data (R Studio, robvis package).

7.1.5. Meta-analysis calculation

Tools such as Meta-essentials,RevMan , or OpenMEE can be used to conduct a meta-analysis for free by reviewers unfamiliar with statistical software or programming language.

Reviewers familiar with coding can benefit from software such as MetaorMetaforpackages and scripts in R, Python, or OpenMeta [Analista]. For example, the metagear package for R is a free resource that, in addition to facilitating data sorting and extraction, is also used for meta-analysis.

Here, we provide SysPhar script templates for metafor (meta-analysis, publication bias) in addition to metapower for calculating statistical power, if applicable.

Phase 5: Publication of the results of the review

9.1 Where and how to publish?

There are different strategies for publishing, sharing, and disseminating review results, including scientific events, workshops, online platforms, and peer-reviewed journals.

In peer-reviewed journals, except for some specifications, the systematic review and meta-analysis report can be organized as other types of publications. Some journals request articles to submit a PRISMA checklist to facilitate peer review.

The publication can be organized as follows:

§ Introduction: theoretical history, hypothesis, and the question of review.

§ Methods: the protocol registration number, research strategy, screening strategy, data extraction, quality assessment, and analysis are briefly presented. Important: Calculations and interpretations of effect sizes must be explicitly stated because they affect the data discussion.

§ Results: description of the results of the search and screening processes of studies, presenting the: PRISMA checklist as a figure of the article; description of the qualities of the publications included in the review and systematization of the results in tables (for the discussion of the external validity of the studies); description of the results of the bias risk analysis of the studies included in the review (for the discussion of the internal validity of the studies); description of the numerical results of the meta-analysis (estimates of overall combined effect size with or without correction by publication bias, estimates of combined effect size stratified by subgroups, heterogeneity of estimates) and graphical presentation, in general, the forest graph is used for meta-analysis and funnel graphs when evaluating the publication bias.

§ Discussion: the evaluation of the internal and external quality of the publications included should be addressed; meta-analysis results (the direction, size, and heterogeneity of combined effect estimates), impact of the quality of studies, the risk of experimental biases, the risk of publication bias, and the heterogeneity of studies on the estimates of the size of the combined effect.

§ Conclusion: the reliability of the review results according to the evaluations of quality, heterogeneity, publication bias, and limitations of the review process can be disclosed.

9 Phase 6: Keeping living systematic review

9.1 Update and publish

The existing guidelines guide live systematic reviews to be published according to models like the Cochrane publications.

The available templates are as follows:

§ New publications with each new update (usually done at annual intervals), obtaining a recent version, and DOI in PubMed. NOTE: This practice is likely to result in numerous publications.

§ A publication where the introduction and methods of the main manuscript have not changed, only the results section has changed.NOTE: The manuscript will require fewer resources to prepare in the second model, but the original version must have been written generically to accommodate new emerging information in subsequent versions.

§ Publications in which the findings of the updated analyses can be presented as new appendices, without alteration in the original abstract or manuscript.NOTE: This model makes it difficult to add or remove authors as changes in their contribution to subsequent versions.

§ Alternatively, the model in which the results of updates are summarized in public domain data repositories, from which they present the results updated visually. NOTE: Ideally, these repositories should be interactive (e.g., app RShiny).

When no recent studies are identified for inclusion, only published research data must be updated. However, if further studies are found, a new publication is needed.

10 About

10.1 SysPhar

SysPhar (Systematic Pharmacology) is a resource produced by CAMARADES BR.

In this SysPhar guide, we gather and organize information, documents, links, and other tools that we find helpful in helping pharmacologists perform their systematic reviews and meta-analysis in pharmacology.

The resources presented in SysPhar are publicly available on the pages of organizations such as CAMARADES, Cochrane, OSF, PROSPERO, SYRCLE, PRISMA, and others. We thank these organizations and teams for making these resources publicly available and for free!

This feature was last updated on: August 07th, 2023.

10.2 How to quote the tool?

Sysphar. Systematic Pharmacology (January 2022), CAMARADES, Brazil. Available at https://camaradesbrasil.bio.br/. Accessed in Jun 20, 2025.

10.3 Our Team

• Juliana Aparecida Bolzan, Ma.
• Cilene Lino de Oliveira, PhD.
• Tamires Martins, Ma.
• Sofia Diehl Döring, PharmB.

10.5 Contact

If you have questions about the features or would you like to ask a specific question, would you like to contribute to the text?

Contact us!