OIS Calculator — Optimal Information Size for GRADE Imprecision

The OIS Calculator computes the Optimal Information Size (OIS) — the number of participants a body of evidence would need in order to detect a specified effect — and compares it with the sample size your meta-analysis has actually accumulated. That comparison is the arithmetic behind the imprecision domain of GRADE. Continuous and binary outcomes are covered, and every calculation runs in your browser.

How to use

Two tabs, one per outcome type. Enter the design parameters and the OIS is returned as a total and per group, with the intermediate quantities. Add the total N currently pooled in your review and the tool reports what percentage of the OIS you have reached and which downgrade that suggests.

• Continuous: choose a mean difference with an SD, or an SMD entered directly; give the minimal clinically important difference (MCID, Δ), α (0.05, 0.01, 0.10, two-sided) and power (80%, 90%, 95%).
• Binary: give the control event rate (CER, %), the effect as a risk ratio, a relative risk reduction (%) or an intervention event rate (%), plus α and power.
• The current total N is optional in both tabs.

With MCID = 0.5, SD = 1.0, α = 0.05 and 80% power, the OIS is 126 in total, 63 per group (implied SMD 0.500, z sum 2.802). A current total N of 100 is 79.4% of that; being at or above 50%, the tool suggests considering a one-level downgrade for serious imprecision. With CER = 20% and RR = 0.75, the OIS is 1,806 in total, 903 per group (implied IER 15.0%, risk difference 5.0%, 316 expected events); a current total N of 400 is 22.1%, below 30%, so two levels are suggested.

The one thing you must bring is a threshold you can defend. The OIS is entirely determined by it, so pre-specify it in your protocol rather than choosing it after seeing the pooled estimate.

Method and primary sources

The calculator uses the standard closed-form two-group sample size formulas under a normal approximation, displayed on screen next to the result so they can be checked by hand:

• Mean difference: n per group = 2 × (z_α/2 + z_β)² × SD² / Δ²
• SMD: n per group = 2 × (z_α/2 + z_β)² / SMD²
• Binary: n per group = (z_α/2 + z_β)² × [p₁(1 − p₁) + p₂(1 − p₂)] / (p₁ − p₂)²
• Defaults: α = 0.05 two-sided gives z = 1.960; 80% power gives z_β = 0.842.

Judging a body of evidence against the sample size an adequately powered trial would require comes from the GRADE guidelines on imprecision [1]. It matters where that judgement sits in the sequence. Under the minimally contextualised approach of GRADE Guidance 34 [2], the primary criterion is the confidence interval, not the information size: you compare the interval with the threshold — the null, in this approach — and rate imprecision on whether it crosses it. The OIS is applied as a secondary consideration, in situations where the interval does not cross the threshold but the relative effect is large and the accumulated sample is small. Read the percentages this calculator returns in that order: interval against threshold first, information size second.

The bands applied to the ratio of your current total N to the OIS are:

• 100% of the OIS or more — no downgrade on this criterion.
• 50% to 100% — consider one level down.
• 30% to 50% — consider one or two levels down.
• Below 30% — consider two levels down.

GRADE Guidance 35 [3] sets out the contextualised alternative, where the comparison is with a decision threshold rather than the null; in that framing, calculate the OIS for your threshold.

The tool notes on screen that the final judgement must integrate the width of the confidence interval with the clinical context, and that is worth repeating: the percentage of the OIS is an input to the imprecision rating, not a substitute for it. If you are also running the pairwise analysis, pmatools applies the same OIS logic inside its GRADE imprecision domain.

How to cite

Cite the GRADE guidance as the methodological source; the calculator is a convenience for the arithmetic. A Methods paragraph might read:

Imprecision was rated using a minimally contextualised approach (GRADE Guidance 34): confidence intervals were compared with the null, and the optimal information size was used as a secondary criterion. The optimal information size was calculated for a minimally important difference of 0.5 SD with α = 0.05 (two-sided) and 80% power, using the OIS Calculator (Furukawa Y. OIS Calculator. https://yukifurukawa.jp/ois-calculator-grade-imprecision/).

Report the MCID, α and power, not only the resulting OIS; without them the number cannot be reproduced. The tool is maintained by Yuki Furukawa, a psychiatrist specialising in insomnia who works on systematic reviews and meta-analysis.

Limitations

• Two arms with equal allocation are assumed; cluster designs, unequal allocation and multi-arm comparisons are not handled.
• The formulas are closed-form with no continuity correction, so binary results with very small event counts are indicative only.
• The percentage thresholds prompt a judgement rather than replace one. Applied mechanically, without the confidence interval and the clinical context, they are a misuse of the tool.
• Results move substantially with the MCID, which is an argument for pre-specification and for a sensitivity analysis.
• Imprecision only. The other GRADE domains have to be rated elsewhere.

References

1. Guyatt GH, et al. GRADE guidelines 6. Rating the quality of evidence—imprecision. J Clin Epidemiol. 2011;64(12):1283-1293. PubMed
2. Zeng L, et al. GRADE Guidance 34: update on rating imprecision using a minimally contextualized approach. J Clin Epidemiol. 2022;150:216-224. PubMed
3. Schünemann HJ, et al. GRADE guidance 35: update on rating imprecision for assessing contextualized certainty of evidence and making decisions. J Clin Epidemiol. 2022;150:225-242. PubMed