ANCOVA

The method, in full

The practical guide behind this tool: when it applies, how to read the output, and the traps that make it say the wrong thing.

Read the guide →

A simulated example, with the code and its output

The numbers below come from made-up data, made by the code on this page. They show the method. They are not output from this tool and not a finding about your data.

Question: Do three groups differ on an outcome once their different baselines are adjusted for?

Inputs: Simulated data, not a real study. Seed 11 makes 40 people in each of groups A, B and C. Built in by construction: outcome = 10 + 0.8 x baseline + a group effect (A 0, B 2.0, C 0.0) + normal noise with sd 3. Baselines differ by group (means 50, 55 and 60, sd 8), so the raw group means mislead. The built-in adjusted differences are B minus A 2.0, C minus A 0.0 and C minus B -2.0.

Method: The steps the page describes, in numpy and scipy. First the equal-slopes check: an F test comparing a model with a separate baseline slope per group against one common slope. If slopes are similar, ANCOVA fits group plus baseline with one common slope, and an F test asks whether the group terms matter after the baseline. Adjusted means are each group's predicted outcome at the overall mean baseline. Pairwise differences come from the same model with t-based 95% intervals. These p-values are not corrected for multiple comparisons. For contrast, a one-way ANOVA ignores the baseline.

Result: Raw group means are A 49.68, B 56.45 and C 58.36. The equal-slopes check gives F(2,114) = 0.08, p = 0.925, so a common slope is reasonable here. After adjusting for baseline the group effect is F(2,116) = 5.78, p = 0.0041, against F(2,117) = 16.81 from the one-way ANOVA that ignores baseline, which overstates the gap. The fitted slope is 0.848 (0.8 built in). Adjusted means at baseline 55.3 are A 54.73, B 55.91 and C 53.84. Pairwise: B minus A is 1.19 (95% CI -0.08 to 2.45), C minus A is -0.88 (-2.28 to 0.51), C minus B is -2.07 (-3.30 to -0.84). Only C minus B excludes zero, though B minus A was built at 2.0, which shows how far one run of 40 per group can land from the truth.

Limit: The data has straight-line baseline effects, equal slopes by construction, normal noise and equal group sizes. It does not show what happens when slopes differ between groups, the baseline is affected by the treatment, or the data is unbalanced. It also does not show how the tool reports its results.

Python script (numpy and scipy only):

import numpy as np
from scipy.stats import f as fdist, t as tdist
# Simulated data. Built in: 3 groups x 40, outcome = 10 + 0.8*baseline + group effect (A 0, B 2.0, C 0.0) + noise sd 3.
# Groups differ in baseline (A mean 50, B mean 55, C mean 60), so raw means mislead.
rng = np.random.default_rng(11)
G = ["A", "B", "C"]; bm = {"A": 50, "B": 55, "C": 60}; eff = {"A": 0.0, "B": 2.0, "C": 0.0}
g = np.repeat(np.arange(3), 40)
x = np.concatenate([rng.normal(bm[k], 8, 40) for k in G])
y = 10 + 0.8 * x + np.array([eff[k] for k in G])[g] + rng.normal(0, 3, 120)
def rss(X): b, *_ = np.linalg.lstsq(X, y, rcond=None); e = y - X @ b; return e @ e, b, X.shape[1]
D = np.eye(3)[g]
print("raw group means:", " ".join("%s %.2f" % (k, y[g == i].mean()) for i, k in enumerate(G)))
# 1. equal slopes check: group x baseline interaction
r_full, _, k_full = rss(np.column_stack([D, D * x[:, None]]))
r_add, b, k_add = rss(np.column_stack([D, x]))
F = ((r_add - r_full) / (k_full - k_add)) / (r_full / (120 - k_full))
print("equal-slopes check: F(%d,%d) = %.2f, p = %.3f" % (k_full - k_add, 120 - k_full, F, fdist.sf(F, k_full - k_add, 120 - k_full)))
# 2. ANCOVA group effect after baseline
r_nog, _, k_nog = rss(np.column_stack([np.ones(120), x]))
F2 = ((r_nog - r_add) / (k_add - k_nog)) / (r_add / (120 - k_add))
print("group effect adjusted for baseline: F(%d,%d) = %.2f, p = %.2g" % (k_add - k_nog, 120 - k_add, F2, fdist.sf(F2, k_add - k_nog, 120 - k_add)))
rawF = (sum(40 * (y[g == i].mean() - y.mean()) ** 2 for i in range(3)) / 2) / (sum(((y[g == i] - y[g == i].mean()) ** 2).sum() for i in range(3)) / 117)
print("one-way ANOVA ignoring baseline: F(2,117) = %.2f, p = %.2g" % (rawF, fdist.sf(rawF, 2, 117)))
# 3. adjusted means at the grand mean baseline, pairwise differences
slope = b[3]; xbar = x.mean(); adj = b[:3] + slope * xbar
print("slope %.3f (built in 0.8); adjusted means at baseline %.1f:" % (slope, xbar), " ".join("%s %.2f" % (k, adj[i]) for i, k in enumerate(G)))
X = np.column_stack([D, x]); cov = (r_add / (120 - k_add)) * np.linalg.inv(X.T @ X); df = 120 - k_add
for i, j in ((1, 0), (2, 0), (2, 1)):
    c = np.zeros(4); c[i] = 1; c[j] = -1; d = c @ b; se = np.sqrt(c @ cov @ c); tt = d / se
    ci = tdist.ppf(0.975, df) * se
    print("%s - %s: diff %.2f, 95%% CI %.2f to %.2f, p %.3g (unadjusted for multiple comparisons)" % (G[i], G[j], d, d - ci, d + ci, 2 * tdist.sf(abs(tt), df)))
print("built-in adjusted differences: B-A 2.00, C-A 0.00, C-B -2.00")

Printed output from running the code above:

raw group means: A 49.68 B 56.45 C 58.36
equal-slopes check: F(2,114) = 0.08, p = 0.925
group effect adjusted for baseline: F(2,116) = 5.78, p = 0.0041
one-way ANOVA ignoring baseline: F(2,117) = 16.81, p = 3.8e-07
slope 0.848 (built in 0.8); adjusted means at baseline 55.3: A 54.73 B 55.91 C 53.84
B - A: diff 1.19, 95% CI -0.08 to 2.45, p 0.0654 (unadjusted for multiple comparisons)
C - A: diff -0.88, 95% CI -2.28 to 0.51, p 0.213 (unadjusted for multiple comparisons)
C - B: diff -2.07, 95% CI -3.30 to -0.84, p 0.00115 (unadjusted for multiple comparisons)
built-in adjusted differences: B-A 2.00, C-A 0.00, C-B -2.00

What's in the report?

Do your groups differ once a baseline is adjusted for? Adjusted means and pairwise differences, plus the equal-slopes check ANCOVA rests on.