pmatools — Pairwise Meta-Analysis with GRADE

pmatools runs a pairwise meta-analysis from study-level data and then walks you through a certainty-of-evidence assessment following the BMJ 2025 Core GRADE series. The app exports the pooled results, the plots, a GRADE evidence profile and summary-of-findings table, and a reproducible R script, in a single ZIP file.

How to use

The app is a four-step wizard: 1 Data → 2 Meta-analysis → 3 GRADE → 4 Export. The version numbers you need for reporting are not on the front page: they are printed at the end of the Text results output in Step 2, where at the time of writing they read pmatools : 0.3.4 (vendored) / meta : 8.3.0 / R : 4.5.1.

Step 1: data

Data must be in long format, one row per study and arm (or per study, outcome and arm if an outcome column is present). Wide format is not accepted.

• Required columns: studlab, treat, n, plus event for binary outcomes or mean and sd for continuous outcomes.
• Three ways to get data in: load one of the built-in sample datasets, upload a .csv or .xlsx file, or paste a block copied from Excel.
• Eligible studies that reported no usable outcome data can be entered with studlab and n only. They are placed in a missing-results subgroup at the GRADE step and appear in the risk-of-bias-due-to-missing-evidence forest plot, so that unreported studies stay visible rather than silently dropping out.

Step 2: meta-analysis

• Effect measure: odds ratio or risk ratio for binary outcomes; mean difference or standardised mean difference for continuous outcomes.
• Model: random effects or common (fixed) effect. Between-study variance estimator: REML or DerSimonian-Laird. A continuity correction is applied to zero cells.
• Pooling method: inverse variance, Mantel-Haenszel, or Peto. The default depends on the effect measure — an odds ratio starts on inverse variance, a risk ratio starts on Mantel-Haenszel — so if you switch measures mid-analysis, check the pooling method before reporting it.
• The Hartung-Knapp adjustment is applied by default, the confidence interval for tau-squared uses the Q-profile method, and a prediction interval is reported alongside the pooled estimate.
• Subgroup analysis is available. Meta-regression is not.

One of the built-in samples is a 17-RCT dataset on response in trials of cognitive behavioural therapy for insomnia in people with major depressive disorder, taken from a published meta-analysis [1]. On that dataset a binary, odds-ratio, random-effects, inverse-variance, REML analysis returns OR 2.3280 [1.6222; 3.3410], prediction interval [0.9075; 5.9722], tau-squared 0.1678, I-squared 36.1% [0.0%; 64.3%], Q = 25.05 on 16 df (p = 0.0690), k = 17.

Step 3: GRADE

Five domains and an overall certainty rating. Each domain shows the algorithm it uses, an automatic suggestion, and a manual override, so the reviewer’s judgement always has the last word.

• Indirectness is entirely manual (default: no concerns); nothing about applicability can be inferred from the data table.
• Imprecision combines two conditions: whether the confidence interval crosses the null, and whether the accrued information reaches the optimal information size (OIS). The target number of events and the target total N can be overridden. A standalone calculator for the same quantity is available at https://yukifurukawa.jp/ois-calculator-grade-imprecision/.
• Publication bias follows the Core GRADE flow: with k of 10 or more, Egger’s test is run (p < 0.01 suggests rating down two levels, 0.01 to < 0.05 one level, p of 0.05 or above no downgrade). Trim-and-fill is displayed for reference only and does not drive the rating.
• The summary-of-findings table reports control and intervention event rates side by side, converts a standardised mean difference to an odds ratio using Chinn’s formula, and lets you switch the prediction interval on or off.

On the sample dataset above the overall rating comes out as Moderate certainty, with inconsistency the only domain rated serious.

Step 4: export

One ZIP file with a selectable set of contents: the long-format CSV actually analysed, a reproducible R script (analysis.R), the results text, the forest plot (PDF and PNG), a risk-of-bias-stratified forest plot, the funnel plot, and the GRADE evidence profile and summary-of-findings tables as .docx. Two further items — the trim-and-fill plot and the publication-bias missing-results forest plot — are only offered when k is 10 or more, the same threshold that governs the publication-bias assessment itself. On screen you also get the forest plot, a contour-enhanced funnel plot, and the verbatim print output of the underlying R object.

Method and primary sources

Pooling is performed with the meta package for R [2], so the estimates are the ones you would obtain by calling metabin() or metacont() directly with the same arguments; the exported analysis.R makes that correspondence explicit and lets a reviewer re-run the analysis outside the app.

The certainty assessment follows the BMJ 2025 Core GRADE series. Two of the underlying methodological references are worth reading before using the imprecision domain: the original GRADE guidance on imprecision [3] and the updated minimally contextualised approach [4], with the contextualised counterpart in [5]. Risk of bias is expected to have been assessed with RoB 2 [6] before the data reach the app, and the inconsistency domain is reported with the I-squared statistic [7].

How to cite

The app displays the following wording, which can be used in a Methods section as it stands:

Pairwise meta-analysis was performed using the {meta} R package (Balduzzi et al. 2019). Certainty of evidence was rated using the GRADE approach following the BMJ 2025 Core GRADE series (Guyatt et al. 2025), implemented in pmatools (Furukawa Y, https://yukifurukawa.jp/pmatools/).

Add the version string from the end of the Step 2 text results and the date of the analysis, and state the model, pooling method, between-study variance estimator and whether the Hartung-Knapp adjustment was used, since all of these are user-selectable and the pooling default changes with the effect measure. The full reference for the meta package is [2].

Limitations

• Risk difference is not available for binary outcomes; only odds ratio and risk ratio.
• No meta-regression. Effect modification can only be explored by subgroup analysis, which is the weaker design for that question.
• No network meta-analysis. The app compares two arms at a time.
• Long format only. Data held in a wide layout must be reshaped first.
• The Hartung-Knapp adjustment is on by default. This is a defensible default but it is a default, not a neutral setting, and it should be reported.
• Trim-and-fill output is descriptive and is deliberately kept out of the publication-bias rating.
• The automatic GRADE judgements are suggestions computed from the data at hand. Indirectness in particular cannot be automated, and every domain can and sometimes should be overridden.
• The app itself states the broader caveat: statistical pooling is only one part of a systematic review. A prespecified protocol, a comprehensive search, duplicate independent screening and data extraction, and risk-of-bias assessment all belong before the analysis, and no tool can supply them afterwards.

References

1. Furukawa Y, et al. Cognitive behavioral therapy for insomnia to treat major depressive disorder with comorbid insomnia: A systematic review and meta-analysis. J Affect Disord. 2024;367:359-366. PubMed 39242039
2. Balduzzi S, Rücker G, Schwarzer G. How to perform a meta-analysis with R: a practical tutorial. Evid Based Ment Health. 2019;22(4):153-160. PubMed 31563865
3. Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines 6. Rating the quality of evidence—imprecision. J Clin Epidemiol. 2011;64(12):1283-1293. PubMed 21839614
4. Zeng L, Brignardello-Petersen R, Hultcrantz M, et al. GRADE guidance 34: update on rating imprecision using a minimally contextualized approach. J Clin Epidemiol. 2022;150:216-224. PubMed 35934265
5. Schünemann HJ, Vernooij RWM, Brignardello-Petersen R, et al. GRADE guidance 35: update on rating imprecision for assessing contextualized certainty of evidence and making decisions. J Clin Epidemiol. 2022;150:225-242. PubMed 35934266
6. Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898. PubMed 31462531
7. Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557-560. PubMed 12958120