ds@mcpanalytics.aiA data scientist you can email your data and question to, and get a reliable response. Not AI slop.A data scientist you can email. Not AI slop.
Worked example · McNemar's test

McNemar's test or chi-square? Paired yes/no data, decided.

The reproducible worked example behind our McNemar lesson: 200 paired approve/deny decisions, agreement, kappa, and the four variants of the test.

The data

200 paired approve/deny decisions from two reviewers on the same cases, generated at seed 42 so it reproduces exactly.

The headline numbers

85% agreement sounds like reassurance. It is not. The 30 disagreements split 24 to 6, which is one reviewer holding a lower bar rather than two people being human.

Download and re-run it

Every figure in the document is derived when it is knitted, not typed in. Re-run the source and you should get the same numbers, or we have a problem worth hearing about.

The honest limit

The test tells you the two judges differ. It cannot tell you which one is correct. The remedy is calibration, not blame.

The brief for an AI agent

If an AI assistant is choosing a method for you, it needs different things than you do: the discriminating condition, the questions this gets mistaken for, the failure modes that return a plausible number instead of an error, and figures it can cite rather than generate. We publish that as its own document, written to one bar: an agent asked whether to run this analysis should be able to answer from it alone, including saying no.

Read the agent brief

Where this came from

Watch the lesson Run it on your own data The brief for an AI agent All worked examples

Your turn

Bring your own data and the question you actually need answered.

CympleData Scientist Send me your data and question, I’ll send you the analytics. ds@mcpanalytics.ai