Where is the biggest leak in your funnel?

That question has two correct answers. They point at different steps, and they fund different work.

2,500 accounts · every figure checked three ways · the data and the code are below

What is funnel analysis?

Funnel analysis measures where people stop in an ordered sequence of steps. Signup, checkout, onboarding, a job application, a support escalation: anything where reaching step three means you already got through steps one and two. It is sometimes called conversion analysis or drop-off analysis, and in older marketing writing a purchase funnel, after the shape it draws.

The idea is old and the arithmetic is simple. What makes it worth doing carefully is that the obvious summary of a funnel is ambiguous, and the ambiguity is invisible: the phrase "the biggest leak" describes two different measurements that usually disagree about which step to fix.

It answers questions a single conversion rate cannot:

• Which step loses the most people, in absolute numbers?
• Which step converts worst, as a share of the people who reached it?
• Does either answer change once you split by channel, plan or device?

One thing it deliberately does not handle: people who are still mid-journey. Someone who signed up yesterday has not failed to convert, they simply have not got there yet, and a funnel counts them as a drop-off. If the timing matters more than the sequence, the tool you want is survival analysis, which is built for exactly that.

The ambiguity in "the biggest leak"

Somebody puts a funnel chart on a slide, points at the widest gap, and a sprint gets funded. The problem is that "biggest" was never defined, and the two available definitions disagree.

The volume answer is which step loses the most accounts. The efficiency answer is which step converts worst, as a proportion of the people who actually reached it. Both are correct. Both are useful. They are usually different steps, and only one of them is the step whose behaviour is genuinely unusual.

Simple explanation 2 min The full lesson 7 min

Both answers, on a worked dataset

Everything below runs on one constructed dataset: a quarter of self-serve signups at a hypothetical B2B software company, with the counts set by hand and the dates drawn from a fixed seed. 2,500 accounts arrive at a plans page, and 180 end up paying. Five steps, one row per account per step reached.

End to end, 7.2% of visitors pay. The volume answer is the first step: 1,450 accounts never start free, more than any other gap loses. The efficiency answer is three steps further down: of the accounts that activate, only 36% ever invite a teammate.

Why the first step almost always wins the volume answer

Worth naming, because it stops the volume answer being over-read: the first gap draws from the largest pool. A step that loses a modest share of 2,500 people will lose more humans than a step that loses a terrible share of 680. So "loses the most" is partly a fact about arithmetic and position, not only about how bad the step is. The efficiency answer is what tells you a step is behaving unusually.

The finding the aggregate was hiding

A rate for the whole funnel is a summary of groups that may behave nothing alike. Split these accounts by acquisition channel and one of them does something the aggregate cannot show.

Partner accounts activate at 76% against paid search's 60%, a gap of nearly 16 points. If the two channels truly activated at the same rate, a gap that large would turn up about three times in a hundred thousand. Then at the next step they invite a teammate at 20% against paid search's 40%, a reversal of 20 points that would turn up about eight times in a million under no true difference. (These accounts are a constructed dataset, so the tests show how the check is run, not a discovery about real customers.)

The aggregate 36% invite rate is an average over a channel running at 20% and one running at 40%. It describes neither. It does not make partner the whole problem: paid search loses more accounts at the invite step (164 against partner's 152, with 119 for direct). It makes partner the most lopsided part of it, and the strongest candidate to investigate first. The counts cannot say why partner accounts do not invite a teammate, or what a fix would cost against the roughly 38 extra invites a quarter it could recover (partner at paid search's invite rate). Those are the questions to answer before funding a sprint.

What the analysis needs from your file

One row per account per step reached. Three columns, plus one optional column that unlocks the most useful part.

Real rows from the file below. One account appears once per step it reached, and simply stops appearing at the step it did not reach. That is how the analysis knows where it fell out; there is no "did not convert" row to add.

1. An account or session id, so steps can be linked into a journey.
2. A stage label, and the steps have to be genuinely ordered. Optional or parallel stages produce a funnel whose narrowing is an artifact of the order you chose.
3. A timestamp for when that step was reached.
4. Something to break it down by, and this is the one worth adding: channel, plan, device. Without it you cannot find a reversal like the one above.

And per-step totals are not enough for the most useful part. If all you have is 2,500 then 1,050 then 680, you can still work out the losses, the step rates and an interval for each, and so tell the volume answer from the efficiency answer. What you cannot do is segment: you cannot find a reversal like the channel one, because totals cannot be turned back into rows.

Every number here, you can re-run

This page is not a screenshot of an analysis. It is an analysis. The document derives every figure when it runs, and it was checked three ways: by the document itself, by R's two-proportion test called directly on each step, and by an independent Python reimplementation. All three agree to every digit shown.

Read the full knitted analysis, the document itself with every chart, table and line of code as it ran. Or take the source and the data below and run it yourself.

The limits, stated plainly

Three things this analysis cannot tell you, and none of them announces itself in the output.

It cannot see people who are still mid-journey. An account that started last week and has not activated yet is counted as a drop-off. Over a short window, or a period where signups grew, that alone can move every rate downward. Funnel analysis has no concept of an unfinished journey; survival analysis does.

It cannot tell you why. The reversal above is a real, measured difference. Whether partner accounts stay single-seat because of who they are, how they were sold, or what the product does after activation is not in this data.

It will draw whatever order you map. The narrowing shape is produced by the sequence you declare, so a mis-declared sequence produces a confident and meaningless funnel.

Download

Two files. The notebook regenerates the dataset from its seed, so re-knitting reproduces this page's figures exactly.

• funnel.Rmd · the analysis, R Markdown source10 KB
• funnel_events.csv · 4,655 events across 2,500 accounts251 KB

Both files are released under CC BY 4.0: use them anywhere, including commercially, with attribution.

Why this matters to the business

three 60-second cuts · scroll for more →

Where should the next sprint go?

Somebody is about to point at the tallest bar on a funnel chart. That is a coin flip wearing a suit, because the tallest bar answers only one of the two questions.

Should we cut the channel that is not converting?

One channel here is the best of three at one step and the worst of three at the next. Cut it on the second number and you lose the first.

Seven percent of visitors buy. Is that good?

End-to-end conversion is fine for a board slide and useless for deciding what to fix, because it never says where the other ninety three went.