MetaLab — Research Synthesis, Power & Bias Workbench

Module 123

Data

Effect measure

Model

Between-study variance τ²

Confidence interval

Sort studies by

Contour-enhanced funnel Trim-and-fill on funnel

Forest plot

Funnel plot

Results

Pooled effect

—

Heterogeneity I²

—

Cochran Q

—

Prediction interval

—

where a new study may fall

Egger's test

—

Trim-and-fill

—

Power curve

Null vs alternative

Simulated experiments

Press “Simulate” to run 2,000 experiments at the current design

significantnot significanttrue effect

Analytic power

—

Required n

—

Empirical power

—

Critical value

—

Mean d | significant

—

Sign errors

—

Underpowered studies that reach significance systematically overestimate the true effect — the “winner’s curse”.

The literature

publishedfile drawerimputed

What would a meta-analysis conclude?

p-curve

significant published p-values

Diagnostics

Published

—

Significant (published)

—

Bias of naive estimate

—

Egger's test

—

How MetaLab works

MetaLab implements the statistics behind evidence synthesis from scratch — distribution functions, estimators and tests — and checks them against reference values (e.g. the BCG results reported by the metafor R package and G*Power's power tables).

1 · Meta-analysis

Each study contributes an effect yᵢ with sampling variance vᵢ (log risk / odds ratio from 2×2 counts, Hedges' g from means and SDs, or given directly). Studies are weighted by inverse variance:

μ̂ = Σ wᵢ yᵢ / Σ wᵢ, wᵢ = 1 / (vᵢ + τ²), SE(μ̂) = 1/√Σwᵢ

Fixed effect: τ² = 0 — one true effect.
DerSimonian–Laird: moment estimator τ² = max(0, (Q − (k−1)) / (Σw − Σw²/Σw)).
REML: restricted maximum likelihood, solved by Fisher scoring.
Knapp–Hartung: rescales the SE and uses a t_k−1 reference — better coverage with few studies.

Heterogeneity: Cochran's Q (χ²_k−1), I² = τ² / (τ² + s̃²) and a prediction interval μ̂ ± t_k−2√(τ² + SE²) for the effect in a new setting.

2 · Small-study effects

Funnel plot — effect vs standard error; contours mark where p < .10, .05 and .01.
Egger's test — regress yᵢ/SEᵢ on 1/SEᵢ; a non-zero intercept signals asymmetry.
Trim-and-fill (Duval & Tweedie, L₀) — iteratively trims the asymmetric tail, re-estimates the centre and imputes mirror-image “missing” studies.
PET — the intercept of a weighted regression of y on SE: the effect of a hypothetical infinitely precise study.

3 · Power

power = P(|T| > t_crit), T ~ t'(ν, δ), δ = d·√(n/2)

The non-central t CDF is integrated numerically over the χ² distribution of the variance estimate; t, χ² and incomplete beta/gamma functions are computed from Lanczos log-gamma and continued fractions. Proportions use the normal approximation, correlations Fisher's z.

4 · Bias lab

Simulates a whole research field: studies with random sizes and true effects θᵢ ~ 𝒩(δ, τ²). Significant positive results are always published; the rest only with the chosen probability. p-hacking lets each study measure several outcomes and report the most significant. Compare what an honest meta-analysis of all studies, a naive meta-analysis of the published ones and bias-corrected methods would conclude.

CSV format

Header row required. Recognised columns: study, year plus either yi, vi (or sei), 2×2 counts tpos, tneg, cpos, cneg, or group summaries m1, sd1, n1, m2, sd2, n2.