# Funnel analysis: agent brief

> **Audience: an agent deciding whether to run this analysis on a dataset.** Not marketing
> copy. Terse, complete, and honest about failure. If you are asked "should I run funnel
> analysis here", you should be able to answer from this file alone, **including saying no**.
>
> Tool: `standard_funnel` · free page: `/free/standard_funnel` ·
> worked example: `/worked-examples/funnel` · lesson: https://youtu.be/jdOLq-7UmSA

---

## 1. What it answers

**Where in an ordered flow people stop, and whether that differs by who they are.**

Signup, checkout, onboarding, application, activation. The distinguishing feature is that
the steps are **ordered and cumulative**: reaching step 3 implies having reached steps 1
and 2. If the "steps" are unordered categories, this is the wrong tool.

**Questions it is mistaken for:**

| Actually asked | Right tool |
|---|---|
| *How long until they churn?* (timing, censored) | `standard_survival` |
| *Do these two groups convert differently?* (one step, two groups) | `standard_proportions` |
| *How do monthly cohorts retain over time?* | `standard_cohort_retention` |
| *Did the redesign change conversion?* (before/after) | `standard_event_impact` |
| *Which accounts will convert?* (per-row prediction) | classification, not this |
| *What is our conversion rate?* | arithmetic. You do not need an analysis for one number |

## 2. When it applies, and when it does not

**Apply it when: the steps are ordered, and you have row-level events showing who reached
which step.**

**Do not apply it when:**

- **You only have per-step totals.** 2,500 → 1,050 → 680 cannot be segmented, cannot carry
  a confidence interval, and cannot distinguish the two "biggest leak" answers. The analysis
  needs rows, and totals cannot be turned back into rows.
- **The steps are not a genuine sequence.** Optional or parallel stages produce a "funnel"
  whose narrowing is an artifact of ordering choices rather than a fact about users.
- **Users can skip steps legitimately.** Then reaching step 3 does not imply step 2, and
  the cumulative reading is wrong.
- **The window truncates journeys.** Accounts that started late in the period have not had
  time to finish. They look like drop-offs and are not. Funnel analysis has no censoring
  concept. This is the one thing it cannot handle and survival analysis can.

## 3. What the data must look like

`column_mapping` requires `user`, `stage`, `timestamp`; `segment_1..3` are optional.

| key | type | meaning |
|---|---|---|
| `user` | any id | the person or account progressing through the flow |
| `stage` | label | which step this row represents |
| `timestamp` | date/datetime | when that step was reached |
| `segment_1..3` | **categorical** | optional breakdowns (channel, plan, device) |

One row per user per step reached. Constraints: min 60 rows, max 100,000.

**Segments must be labels, not measurements.** Values are coerced to text and any level
with fewer than 30 users is suppressed, so a continuous column (seats, MRR, company size)
becomes one group per user, every group is suppressed, and the card reports "no level had at
least 30 users", which blames your sample size when the real problem is the column's shape.
**Bucket a numeric driver into named bands first.** (Filed as LAT-2393; the tool does not
auto-bin and does not name the cause.)

**Also worth knowing:** the three-segment cap is in the schema only. The R discovers segment
columns generically, so the analysis itself is not limited to three.

## 4. What it returns, and how to read each piece

| Output | Read it as | The trap |
|---|---|---|
| **Step counts** | how many reached each step | cumulative, not exclusive |
| **Loss per gap** | how many were lost between two steps | the first gap usually wins by arithmetic, because it has the largest pool to lose from |
| **Step conversion rate** | of those who reached step N, what share reached N+1 | this is the *efficiency* answer and it usually names a different step than the loss column |
| **Confidence interval per rate** | the range the rate could plausibly be | **check overlap before ranking steps.** Two rates whose intervals overlap are not reliably ordered |
| **Segmented funnel** | the same funnel per group | an aggregate rate can be an average of two groups that behave in opposite directions |
| **Time between steps** | how long the move takes | says nothing about who never moved |

## 5. How it fails

**The headline failure is a category error, not an arithmetic one.** "Where is the biggest
leak" is **two questions**: which step loses the most people (volume) and which step
converts worst (efficiency). They routinely name different steps, both answers are correct,
and they fund different work. An analysis that reports only one has silently answered a
question nobody asked.

**Failure modes that yield a plausible wrong answer rather than an error:**

- **Ranking steps on point estimates alone.** If the intervals overlap, the ordering is not
  established, and a sprint gets funded on noise.
- **Simpson's-paradox-shaped segment reversals.** In the worked example the partner channel
  is the *best* of three at activation and the *worst* at the next step. The aggregate hides
  both. Cutting the channel on the aggregate destroys the thing it was best at.
- **Truncated journeys read as drop-off** (§2). Nothing in the output flags it.
- **A continuous segment column** silently produces "not enough data" (§3).
- **Reordering the stages changes the story** and the tool will happily draw whatever order
  you map.

## 6. Verified numbers you may cite

From `worked-example-funnel/VALIDATION.md`, agreed three ways (the notebook's R run, R's
`prop.test` per step, and an independent Python reimplementation).

| Step | Reached | Lost here | Step conversion | 95% CI (pp) |
|---|---|---|---|---|
| visited_pricing | 2,500 | - | - | - |
| → started_trial | 1,050 | **1,450** | 42.0000% | 40.0592–43.9656 |
| → activated | 680 | 370 | 64.7619% | 61.7751–67.6394 |
| → invited_teammate | 245 | 435 | **36.0294%** | 32.4373–39.7817 |
| → converted_paid | 180 | 65 | 73.4694% | 67.3956–78.7921 |

End to end **7.2000%**. Most lost: `visited→trial` (1,450). Worst rate:
`activated→invited` (36.0294%). **Different steps**, and their intervals do **not** overlap
(40.06–43.97 vs 32.44–39.78), so that ranking is real.

**The channel reversal:** partner is best at activation (76.0000% vs paid_search 60.0877%,
+15.91pp, p=2.95111e-05) and worst at invite (20.0000% vs 40.1460%, −20.15pp,
p=7.56137e-06). The aggregate 36.03% invite rate is an average over a channel at 20% and one
at 40%.

**Do not say** that the first step is "the problem" because it loses the most. Losing the
most is partly arithmetic: the first gap draws from the largest pool.

## 7. Where everything is

| | |
|---|---|
| Tool | `standard_funnel` |
| Free page | https://mcpanalytics.ai/free/standard_funnel |
| Worked example | https://mcpanalytics.ai/worked-examples/funnel |
| Dataset | `/worked-examples/files/funnel_events.csv` |
| Notebook / source | `/worked-examples/files/funnel.html` · `funnel.Rmd` |
| Validation record | `lattice/v2/refs/LAT-2369-rmd-lesson-ladder/worked-example-funnel/VALIDATION.md` |

## 8. Open caveat, stated rather than hidden

**No live-tool reproduction has been recorded for this worked example.** `standard_funnel`
returned `Module Bug` on this schema-valid event log (LAT-2373). Every figure above is
agreed three ways *locally*: the notebook, R's canonical functions, and an independent
Python rewrite. None has been reproduced through the deployed tool. Until LAT-2373
closes and an `rpt_` id is recorded, **do not attribute these numbers to a platform run**.

## 9. Routing shortcut

```
Are the steps ordered and cumulative?
├── no → NOT this. Unordered categories are not a funnel.
└── yes
    ├── do you have one row per user per step, with a timestamp?
    │   ├── no, only per-step totals → cannot run. Ask for the event log.
    │   └── yes
    │       ├── could late starters still be mid-journey?
    │       │   └── yes → the tail steps understate. Consider survival analysis instead.
    │       └── want a breakdown? the segment column must be CATEGORICAL. Bucket numbers first.
    └── report BOTH answers: which step loses the most, and which converts worst.
```