Standard Did — MCP Analytics Report

Executive Summary

Did the change move Support Tickets? Treatment vs Control, before vs after.

Observations

120

DiD Effect

-6.585

95% CI

-8.85 to -4.32

p-value

< 0.001

Effect vs Counterfactual

-10.55%

Parallel Trends

not testable

The change is associated with a drop in Support Tickets of 6.58 (95% CI -8.85 to -4.32, p < 0.001) — Treatment moved by -3.22 while Control moved by only 3.36. Relative to the level expected without the change (62.41), that is -10.55%. With a single pre period the parallel-trends assumption could not be tested — the causal reading rests on it untested. This is an observational comparison, not a randomized experiment.

Suggested Interpretation

The short answer

The revamp is associated with a reduction of 6.58 support tickets per customer (95% CI −8.85 to −4.32, p < 0.001), or about 10.55% relative to the expected level of 62.41 without it. The parallel-trends assumption—that the two groups would have moved identically absent the revamp—could not be tested with a single pre period, so the causal reading is not fully secured.

The detail

The difference-in-differences estimate is −6.585 with a 95% confidence interval of −8.85 to −4.32 and p < 0.001. The effect relative to counterfactual is −10.55%. This is an observational design; causal language is licensed only under the untested assumption that the treated and control cohorts would have followed parallel trends without the revamp.

What this can't tell you

With only one pre period, the parallel-trends assumption cannot be verified against historical data. If the two groups were already diverging before the revamp, the estimate would be biased.

Overview

Analysis Overview

Difference-in-differences on Support Tickets: Treatment (treated) vs Control (control), before vs after.

N Observations120

Did Estimate-6.585

Treated Change-3.223

Control Change3.362

Suggested Interpretation

The short answer

The control group's before-to-after movement (3.36 tickets) is subtracted from the treated group's (-3.22 tickets) to isolate the change's effect. This removes confounding trends, seasonality, and shocks that both groups experienced. The resulting difference-in-differences estimate of -6.58 represents the effect attributable to the onboarding revamp, assuming the two groups would have moved identically absent the intervention.

The detail

The treated group moved from 59.046 to 55.823 tickets (a change of -3.223), while the control group moved from 49.422 to 52.783 (a change of 3.362). The difference between these two movements is -6.585. This arithmetic removes any common shock or trend; what remains is the causal effect under the parallel-trends assumption—that without the revamp, Treatment would have drifted like Control.

What this can't tell you

This design cannot rule out group-specific shocks or violations of parallel trends. With only one pre-period, the parallel-trends assumption is not testable, so the causal claim rests on an untested assumption about pre-change trajectories.

Data Preparation

How the groups and periods were identified, and what was dropped.

Initial Rows120

Final Rows120

Rows Removed0

Suggested Interpretation

The short answer

All 120 rows were used; none were dropped. The data was restructured to one pre and one post value per customer, creating a clean 2×2 table (Treatment and Control, each measured before and after) suitable for the difference-in-differences arithmetic.

The detail

Treatment and Control were identified by name; Before and After were read from period labels. The 120 observations were then collapsed to 120 customer-level units (one pre mean and one post mean per customer), which prevents the significance test from being inflated by repeated measurements of the same unit.

What this can't tell you

The preprocessing answers only how the data was structured. It does not verify that the treatment assignment was unbiased or that the two groups were comparable in unobserved ways before the change.

Visualization

The 2x2: Group Means Before and After

Mean Support Tickets per group, before and after the change.

Suggested Interpretation

The short answer

The treated group fell by 3.22 tickets (from 59.05 to 55.82), while the control group rose by 3.36 (from 49.42 to 52.78). The gap between those two movements—6.58 tickets—is the estimate. Do not read the level difference between groups (treated is higher throughout); read only the difference in their slopes.

The detail

Treatment pre: 59.046; Treatment post: 55.823; change = −3.223. Control pre: 49.422; Control post: 52.783; change = 3.362. The difference-in-differences is −3.223 − 3.362 = −6.585. The level gap between groups (about 10 tickets) is expected and is differenced away by the design.

What this can't tell you

The level difference between groups does not identify the effect; only the difference in their movements does. If the groups differed in unobserved ways before the change, the estimate remains biased even if the 2×2 means are accurate.

Visualization

Treated vs Control Over Time

Mean Support Tickets per period for Treatment and Control.

Suggested Interpretation

The short answer

The data carries before/after labels rather than dates, so no timeline can be drawn. With dated observations, you would see whether the two groups' ticket counts ran in parallel before the revamp—the visual foundation of the parallel-trends assumption.

The detail

The period column is labeled 'Phase' (Before/After) rather than carrying specific dates. This prevents a time-series chart that would show both groups' trends and allow visual inspection of whether they were parallel before the change.

What this can't tell you

A dated timeline would have been the clearest way to verify the parallel-trends assumption visually. Its absence means the assumption rests entirely on the statistical design and domain knowledge, not on observed historical alignment.

Data Table

The Difference-in-Differences Estimate

Cell means, per-group changes, and the estimate with its confidence interval.

Measure	Estimate	Detail
Treatment — pre mean	59.05	Mean Support Tickets for Treatment before the change
Treatment — post mean	55.82	Mean Support Tickets for Treatment after the change
Treatment — change	-3.223	How much Treatment moved (-3.22)
Control — pre mean	49.42	Mean Support Tickets for Control before the change
Control — post mean	52.78	Mean Support Tickets for Control after the change
Control — change	3.362	How much Control moved anyway (3.36)
Difference-in-differences	-6.585	95% CI -8.85 to -4.32; p < 0.001
Effect vs counterfactual	-10.55	Relative to the expected level of 62.41 absent the change (-10.55%)

Suggested Interpretation

The short answer

The treated group's 3.22-ticket drop is not the full story: the control group rose 3.36 tickets on its own. The difference between these movements—6.58 tickets—is the revamp's estimated effect. The 95% confidence interval (−8.85 to −4.32) excludes zero, so the difference is highly significant (p < 0.001).

The detail

Treatment pre = 59.046, post = 55.823, change = −3.223. Control pre = 49.422, post = 52.783, change = 3.362. Difference-in-differences = −6.585 (95% CI −8.85 to −4.32, p < 0.001). Relative to the counterfactual level of 62.41, the effect is −10.55%.

What this can't tell you

Statistical significance does not guarantee the estimate is causal. The confidence interval is tight, but it reflects sampling variability, not bias from unobserved confounding or violations of the parallel-trends assumption.

Data Table

Method & Assumptions

The design, the identification choices made, and the assumptions the causal read rests on.

Item	Detail
Design	Observational difference-in-differences: the treated group's before-to-after change minus the control group's. Not a randomized experiment.
Treated group	'Treatment' was identified as the treated group from its name; 'Control' is the control.
Period split	'Before' was read as the pre period and 'After' as the post period from their labels.
Model	OLS regression of Support Tickets on group, period, and their interaction — the interaction term is the difference-in-differences estimate.
Inference	Observations were first averaged to one pre and one post value per Customer ID (120 units), which guards the significance test against repeated measurements of the same unit.
Parallel trends	With only one pre period per group in 'Phase', the parallel-trends assumption cannot be tested here — the causal reading rests on it untested.
Key assumptions	No other shock hit only one group at the same time as the change; group composition is stable; absent the change, the treated group would have moved like the control group.

Suggested Interpretation

The short answer

This is an observational difference-in-differences design, not a randomized experiment. The estimate comes from OLS regression of support tickets on group, period, and their interaction. The causal reading rests entirely on the parallel-trends assumption—that absent the revamp, the treated group would have moved like the control group—which cannot be tested with only one pre period.

The detail

Treated and control groups were identified by name. Before/After periods were read from labels. Observations were collapsed to one pre and one post value per customer (120 units) to guard inference against repeated measurements. The difference-in-differences is the interaction coefficient in an OLS model. The parallel-trends assumption is untestable here because there is only one pre period per group.

What this can't tell you

The design assumes no other shock hit only one group at the same time as the revamp, that group composition is stable, and that treated and control would have trended identically absent the change. These assumptions are not empirically verified.

Methodology

Statistical methodology and diagnostics for Difference-in-Differences

Statistical Method

Difference-in-Differences

Standard-library analysis: did the change cause the difference? Map an outcome, a treated/control group column, and a pre/post period column (labels or dates) — and get the classic 2x2 difference-in-differences: how much the treated group moved, how much the control group moved anyway, and the difference between those two changes with a confidence interval and p-value. When your period column carries dates with several pre-change periods, the report also draws the treated-vs-control trends and runs a parallel-trends check — the assumption the whole causal read rests on.

Data

N = 120 observations

Assumptions

Parallel trends: absent the change, the treated group would have moved like the control group
Exactly two groups — one treated, one control — and a clean before/after split of the periods
No other shock hit only one of the groups at the same time as the change
Group composition is stable — the same kinds of units are measured before and after

Limitations

This is an observational design, not a randomized experiment — the estimate is causal only if the parallel-trends assumption holds
With a single pre-period the parallel-trends assumption cannot be tested, only assumed
Standard errors are classical OLS; without a unit column, repeated measurements of the same unit can overstate precision
If neither group value names a treated arm, the smaller group is assumed treated — if that guess is wrong the estimate's sign flips

Software & Citation

MCP Analytics · mcpanalytics.ai

Code Appendix

Analysis Code

Complete R source code for this analysis

Core Analysis Pipeline

compute_shared <- function(df, params, col_map = list()) {
  # === SHARED EXPORTS ===
  #   initial_rows/final_rows/rows_removed  $ row accounting
  #   n_dropped            $ rows dropped for missing outcome/group/period
  #   outcome_h/group_h/period_h/unit_h  $ humanized user names for prose
  #   trt_lvl/ctl_lvl/trt_rule  $ which group value is treated + how decided
  #   period_rule          $ how periods were split into pre and post
  #   has_time/date_iso    $ whether a time axis exists + ISO strings (or NULL)
  #   n_pre_periods/n_post_periods  $ distinct periods each side of the split
  #   m_t_pre/m_t_post/m_c_pre/m_c_post  $ the 2x2 cell means (analysis frame)
  #   chg_t/chg_c          $ per-group pre-to-post change
  #   did/did_se/did_p/ci_low/ci_high  $ the DiD estimate (OLS interaction)
  #   cf/pct/pct_txt       $ counterfactual level + effect as % of it
  #   collapse_note/n_units  $ unit-collapse behavior (if unit mapped)
  #   pt_testable/pt_slope/pt_p/pt_short/pt_verdict  $ parallel-trends check
  #   cell_means_df        $ period_label, mean_outcome, group_label (4 rows)
  #   trends_df            $ period_date (ISO), mean_outcome, group_label
  #   did_table_df         $ measure, estimate, detail
  #   methods_df           $ item, detail
  #   metrics / json_output
  # === /SHARED EXPORTS ===

Step 1: Resolve mapped columns (humanized for all prose)

initial_rows <- nrow(df)
  outcome_h <- humanize_semantic("outcome", col_map)
  group_h   <- humanize_semantic("group", col_map)
  period_h  <- humanize_semantic("period", col_map)
  unit_h    <- humanize_semantic("unit", col_map)
  for (req in c("outcome", "group", "period")) {
    if (!(req %in% names(df))) {
      stop(sprintf("Difference-in-differences needs &#x27;%s' (%s) mapped.",
                   humanize_semantic(req, col_map),
                   c(outcome = "the numeric outcome",
                     group = "the treated/control indicator",
                     period = "the pre/post indicator or date")[[req]]))
    }
  }
  has_unit <- "unit" %in% names(df)

Step 2: Coerce the outcome to numeric (95% rule)

y_raw <- df$outcome
  if (!is.numeric(y_raw)) {
    ch <- as.character(y_raw)
    non_blank <- !is.na(ch) & trimws(ch) != ""
    conv <- suppressWarnings(as.numeric(ch))
    if (sum(non_blank) == 0 ||
        sum(!is.na(conv[non_blank])) < 0.95 * sum(non_blank)) {
      stop(sprintf("The outcome column &#x27;%s' does not look numeric — fewer than 95%% of its values parse as numbers. Pick a numeric column.",
                   outcome_h))
    }
    y_raw <- conv
  }

Step 3: Keep rows complete on outcome, group, and period (reported)

g_raw <- trimws(as.character(df$group))
  p_raw <- trimws(as.character(df$period))
  keep <- !is.na(y_raw) & !is.na(g_raw) & g_raw != "" & !is.na(p_raw) & p_raw != ""
  n_dropped <- sum(!keep)
  y <- y_raw[keep]; g <- g_raw[keep]; p <- p_raw[keep]
  u <- if (has_unit) trimws(as.character(df$unit))[keep] else NULL
  final_rows <- length(y)
  if (final_rows < 12) {
    stop(sprintf("Only %d usable rows remained after dropping rows missing %s, %s, or %s — at least 12 are needed for a difference-in-differences analysis.",
                 final_rows, outcome_h, group_h, period_h))
  }

Step 4: Identify the treated and control groups

g_lvls <- sort(unique(g))
  if (length(g_lvls) < 2) {
    stop(sprintf("The group column &#x27;%s' has only one distinct value ('%s') — difference-in-differences needs both a treated and a control group.",
                 group_h, g_lvls[1]))
  }
  if (length(g_lvls) > 2) {
    stop(sprintf("The group column &#x27;%s' has %d distinct values (%s) — difference-in-differences needs exactly two: a treated group and a control group.",
                 group_h, length(g_lvls),
                 paste(head(g_lvls, 5), collapse = ", ")))
  }
  low_g <- tolower(g_lvls)
  trt_subs <- c("treat", "test", "interven", "pilot", "exposed", "variant", "trial")
  ctl_subs <- c("control", "comparison", "holdout", "placebo", "untreated", "baseline", "standard")
  hits_trt <- sapply(low_g, function(s)
    any(sapply(trt_subs, function(k) grepl(k, s, fixed = TRUE))) || s %in% c("1", "yes", "true"))
  hits_ctl <- sapply(low_g, function(s)
    any(sapply(ctl_subs, function(k) grepl(k, s, fixed = TRUE))) || s %in% c("0", "no", "false"))
  if (sum(hits_trt & !hits_ctl) == 1) {
    trt_lvl <- g_lvls[hits_trt & !hits_ctl]
    trt_rule <- sprintf("&#x27;%s' was identified as the treated group from its name; '%s' is the control.",
                        trt_lvl, setdiff(g_lvls, trt_lvl))
  } else if (sum(hits_ctl & !hits_trt) == 1) {
    ctl_named <- g_lvls[hits_ctl & !hits_trt]
    trt_lvl <- setdiff(g_lvls, ctl_named)
    trt_rule <- sprintf("&#x27;%s' was identified as the control group from its name; '%s' is therefore treated.",
                        ctl_named, trt_lvl)
  } else {
    n_by_lvl <- table(g)
    trt_lvl <- names(n_by_lvl)[which.min(n_by_lvl)]
    trt_rule <- sprintf("Neither value of &#x27;%s' names a treated or control group, so the smaller group ('%s', %d rows) was ASSUMED to be treated — if that is wrong, the estimate's sign flips.",
                        group_h, trt_lvl, min(n_by_lvl))
  }
  ctl_lvl <- setdiff(g_lvls, trt_lvl)

Step 5: Split periods into pre and post (labels, dates, or numbers)

parse_dates_vec <- function(ch) {
    ok <- function(dd) sum(!is.na(dd)) >= 0.95 * length(ch)
    d <- suppressWarnings(as.Date(ch, format = "%Y-%m-%d"))
    if (!ok(d)) d <- suppressWarnings(as.Date(lubridate::mdy(ch, quiet = TRUE)))
    if (!ok(d)) d <- suppressWarnings(as.Date(lubridate::dmy(ch, quiet = TRUE)))
    if (!ok(d)) d <- suppressWarnings(as.Date(lubridate::ymd(ch, quiet = TRUE)))
    if (ok(d) && length(unique(stats::na.omit(d))) >= 2) d else NULL
  }
  parse_date_one <- function(s) {
    s <- trimws(as.character(s))
    d <- suppressWarnings(as.Date(s, format = "%Y-%m-%d"))
    if (is.na(d)) d <- suppressWarnings(as.Date(lubridate::mdy(s, quiet = TRUE)))
    if (is.na(d)) d <- suppressWarnings(as.Date(lubridate::dmy(s, quiet = TRUE)))
    if (is.na(d)) d <- suppressWarnings(as.Date(lubridate::ymd(s, quiet = TRUE)))
    if (is.na(d)) {
      ny <- suppressWarnings(as.numeric(s))
      if (!is.na(ny) && ny == round(ny) && ny >= 1900 && ny <= 2100)
        d <- as.Date(sprintf("%04d-01-01", ny))
    }
    d
  }

  tv <- NULL          # numeric time value per row (NULL when labels)
  date_iso <- NULL    # ISO date string per row (NULL when no real dates)
  period_kind <- "label"
  dts <- parse_dates_vec(p)
  if (!is.null(dts)) {
    period_kind <- "date"
    tv <- as.numeric(dts)
    date_iso <- format(dts, "%Y-%m-%d")
  } else {
    pn <- suppressWarnings(as.numeric(p))
    if (sum(!is.na(pn)) >= 0.95 * length(p) &&
        length(unique(stats::na.omit(pn))) >= 2) {
      pn_ok <- stats::na.omit(pn)
      if (all(pn_ok == round(pn_ok)) && all(pn_ok >= 1900 & pn_ok <= 2100) &&
          length(unique(pn_ok)) > 2) {
        # Integer years -> treat as annual dates
        dts <- as.Date(ifelse(is.na(pn), NA, sprintf("%04d-01-01", pn)),
                       format = "%Y-%m-%d")
        period_kind <- "date"
        tv <- as.numeric(dts)
        date_iso <- format(dts, "%Y-%m-%d")
      } else if (length(unique(pn_ok)) == 2) {
        period_kind <- "binary_num"
        tv <- pn
      } else {
        period_kind <- "numeric"
        tv <- pn
      }
    }
  }
  if (period_kind != "label" && any(is.na(tv))) {
    keep2 <- !is.na(tv)
    n_dropped <- n_dropped + sum(!keep2)
    y <- y[keep2]; g <- g[keep2]; p <- p[keep2]; tv <- tv[keep2]
    if (!is.null(date_iso)) date_iso <- date_iso[keep2]
    if (!is.null(u)) u <- u[keep2]
    final_rows <- length(y)
  }
  rows_removed <- initial_rows - final_rows

  ts_param <- params$treatment_start %||% params$treatment_date %||% NULL
  if (period_kind == "date") {
    if (!is.null(ts_param)) {
      tsd <- parse_date_one(ts_param)
      if (is.na(tsd)) {
        stop(sprintf("The treatment_start parameter(&#x27;%s') could not be read as a date.",
                     as.character(ts_param)))
      }
      is_post <- tv >= as.numeric(tsd)
      period_rule <- sprintf("Periods were split at the provided treatment start(%s): dates on or after it count as post.",
                             format(tsd, "%Y-%m-%d"))
    } else {
      cut_v <- min(tv) + (max(tv) - min(tv)) / 2
      is_post <- tv > cut_v
      period_rule <- sprintf("No treatment start date was provided, so periods were split at the midpoint of the observed date range(%s to %s) — an assumption; provide a treatment_start parameter for a precise split.",
                             format(as.Date(min(tv), origin = "1970-01-01"), "%Y-%m-%d"),
                             format(as.Date(max(tv), origin = "1970-01-01"), "%Y-%m-%d"))
    }
  } else if (period_kind == "binary_num") {
    v_hi <- max(tv)
    is_post <- tv == v_hi
    period_rule <- sprintf("&#x27;%s' has two numeric values: %s was read as the pre period and %s as the post period (smaller value first).",
                           period_h, r2(min(tv)), r2(v_hi))
  } else if (period_kind == "numeric") {
    if (!is.null(ts_param) && !is.na(suppressWarnings(as.numeric(ts_param)))) {
      cut_v <- as.numeric(ts_param)
      is_post <- tv >= cut_v
      period_rule <- sprintf("Numeric periods were split at the provided treatment start(%s): values on or after it count as post.", r2(cut_v))
    } else {
      cut_v <- min(tv) + (max(tv) - min(tv)) / 2
      is_post <- tv > cut_v
      period_rule <- sprintf("No treatment start was provided, so the numeric periods were split at the midpoint of their range(%s) — an assumption; provide a treatment_start parameter for a precise split.", r2(cut_v))
    }
  } else {
    p_lvls <- unique(p)
    if (length(p_lvls) == 1) {
      stop(sprintf("The period column &#x27;%s' has only one distinct value ('%s') — difference-in-differences needs data from both before and after the change.",
                   period_h, p_lvls[1]))
    }
    if (length(p_lvls) > 2) {
      stop(sprintf("The period column &#x27;%s' has %d distinct non-date values — use a two-level before/after indicator or a date column.",
                   period_h, length(p_lvls)))
    }
    low_p <- tolower(p_lvls)
    pre_subs  <- c("pre", "before", "baseline", "prior", "begin", "start", "old", "early")
    post_subs <- c("post", "after", "follow", "end", "late", "new")
    is_pre_kw <- sapply(low_p, function(s)
      any(sapply(pre_subs, function(k) grepl(k, s, fixed = TRUE))) || s %in% c("0", "no", "false"))
    is_post_kw <- sapply(low_p, function(s)
      any(sapply(post_subs, function(k) grepl(k, s, fixed = TRUE))) || s %in% c("1", "yes", "true"))
    pre_lvl <- NULL
    if (sum(is_pre_kw & !is_post_kw) == 1) {
      pre_lvl <- p_lvls[is_pre_kw & !is_post_kw]
    } else if (sum(is_post_kw & !is_pre_kw) == 1) {
      pre_lvl <- setdiff(p_lvls, p_lvls[is_post_kw & !is_pre_kw])
    }
    if (is.null(pre_lvl) || length(pre_lvl) != 1) {
      stop(sprintf("Could not tell which value of &#x27;%s' is the before period ('%s' vs '%s') — use labels like pre/post or before/after, or a date column.",
                   period_h, p_lvls[1], p_lvls[2]))
    }
    is_post <- p != pre_lvl
    period_rule <- sprintf("&#x27;%s' was read as the pre period and '%s' as the post period from their labels.",
                           pre_lvl, setdiff(p_lvls, pre_lvl))
  }

  if (!any(is_post)) {
    stop(sprintf("All %d usable rows of &#x27;%s' fall before the change — there is no post-period data to compare.",
                 final_rows, period_h))
  }
  if (all(is_post)) {
    stop(sprintf("All %d usable rows of &#x27;%s' fall after the change — difference-in-differences needs pre-change (baseline) data for both groups.",
                 final_rows, period_h))
  }

  trt01 <- as.integer(g == trt_lvl)
  post01 <- as.integer(is_post)
  has_time <- !is.null(tv)
  n_pre_periods <- if (has_time) length(unique(tv[post01 == 0])) else 1L
  n_post_periods <- if (has_time) length(unique(tv[post01 == 1])) else 1L

Step 6: Analysis frame — optionally collapse to unit x period means

af <- data.frame(y = y, trt = trt01, post = post01)
  collapse_note <- ""
  n_units <- NA_integer_
  if (has_unit) {
    agg <- stats::aggregate(list(y = af$y),
                            by = list(unit = u, trt = af$trt, post = af$post),
                            FUN = mean)
    af <- data.frame(y = agg$y, trt = agg$trt, post = agg$post)
    n_units <- length(unique(u))
    collapse_note <- sprintf("Observations were first averaged to one pre and one post value per %s(%d units), which guards the significance test against repeated measurements of the same unit.",
                             unit_h, n_units)
  }
  cell_n <- table(factor(af$trt, levels = 0:1), factor(af$post, levels = 0:1))
  for (ti in c("1", "0")) {
    for (pi in c("0", "1")) {
      if (cell_n[ti, pi] < 2) {
        lvl_name <- if (ti == "1") trt_lvl else ctl_lvl
        stop(sprintf("Group &#x27;%s' has only %d observation(s) in the %s period — at least 2 observations are needed in each of the four %s-by-%s cells.",
                     lvl_name, cell_n[ti, pi],
                     if (pi == "1") "post" else "pre",
                     group_h, period_h))
      }
    }
  }
  if (nrow(af) < 8) {
    stop(sprintf("Only %d observations remained in the analysis frame — at least 8 are needed to estimate the difference-in-differences model.",
                 nrow(af)))
  }

Step 7: The 2x2 cell means and the OLS interaction estimate

cm <- tapply(af$y, list(factor(af$trt, levels = 0:1),
                          factor(af$post, levels = 0:1)), mean)
  m_c_pre  <- cm["0", "0"]; m_c_post <- cm["0", "1"]
  m_t_pre  <- cm["1", "0"]; m_t_post <- cm["1", "1"]
  chg_t <- m_t_post - m_t_pre
  chg_c <- m_c_post - m_c_pre

  fit <- stats::lm(y ~ trt * post, data = af)
  sm <- summary(fit)$coefficients
  if (!("trt:post" %in% rownames(sm)) || is.na(coef(fit)["trt:post"])) {
    stop("The difference-in-differences model could not be estimated — the group and period columns may be confounded(e.g. one group observed only pre, the other only post).")
  }
  did    <- unname(coef(fit)["trt:post"])
  did_se <- sm["trt:post", "Std. Error"]
  did_p  <- sm["trt:post", 4]
  ci <- stats::confint(fit)["trt:post", ]
  ci_low <- unname(ci[1]); ci_high <- unname(ci[2])

Step 8: Effect size in outcome units and as % of the counterfactual

cf <- m_t_pre + chg_c
  pct <- if (abs(cf) > 1e-8) 100 * did / abs(cf) else NA_real_
  pct_txt <- if (is.na(pct)) "not defined(counterfactual level near zero)" else paste0(r2(pct), "%")

Step 9: Parallel-trends check (needs a time axis + >= 2 pre periods)

pt_testable <- FALSE
  pt_slope <- NA_real_; pt_p <- NA_real_
  if (has_time) {
    pre_i <- post01 == 0
    if (length(unique(tv[pre_i])) >= 2 && length(unique(g[pre_i])) == 2) {
      t_idx <- match(tv, sort(unique(tv)))
      dpre <- data.frame(y = y[pre_i], t = t_idx[pre_i], trt = trt01[pre_i])
      pfit <- tryCatch(stats::lm(y ~ t * trt, data = dpre), error = function(e) NULL)
      if (!is.null(pfit)) {
        pco <- summary(pfit)$coefficients
        if ("t:trt" %in% rownames(pco) && !is.na(pco["t:trt", 1])) {
          pt_slope <- pco["t:trt", 1]
          pt_p <- pco["t:trt", 4]
          pt_testable <- TRUE
        }
      }
    }
  }
  if (!pt_testable) {
    pt_short <- "not testable"
    pt_verdict <- sprintf("With only one pre period per group in &#x27;%s', the parallel-trends assumption cannot be tested here — the causal reading rests on it untested.",
                          period_h)
  } else if (pt_p >= 0.05) {
    pt_short <- "consistent"
    pt_verdict <- sprintf("Across the %d pre-change periods the two groups&#x27; trends do not detectably diverge (slope difference %s per period, %s) — consistent with, though not proof of, parallel trends.",
                          n_pre_periods, r3(pt_slope), fmt_pp(pt_p))
  } else {
    pt_short <- "divergent"
    pt_verdict <- sprintf("The two groups were already trending apart before the change(slope difference %s per period, %s) — the difference-in-differences estimate is likely biased, and any causal conclusion should be treated with strong caution.",
                          r3(pt_slope), fmt_pp(pt_p))
  }

Your data has more stories to tell. Run any analysis on your own data — validated R modules, interactive reports, AI insights, and PDF export. 500 free credits on signup.

Try Free — No Signup Sign Up Free

Something wrong? Report it and we'll fix it — or get your credits back

Download PDF · Print · Share ·Refresh this analysis ·Edit this report

What's wrong with this card?

Executive Summary

The short answer

The detail

What this can't tell you

Analysis Overview

The short answer

The detail

What this can't tell you

Data Preparation

The short answer

The detail

What this can't tell you

The 2x2: Group Means Before and After

The short answer

The detail

What this can't tell you

Treated vs Control Over Time

The short answer

The detail

What this can't tell you

The Difference-in-Differences Estimate

The short answer

The detail

What this can't tell you

Method & Assumptions

The short answer

The detail

What this can't tell you

Methodology

Statistical Method

Analysis Code

Core Analysis Pipeline

Step 1: Resolve mapped columns (humanized for all prose)

Step 2: Coerce the outcome to numeric (95% rule)

Step 3: Keep rows complete on outcome, group, and period (reported)

Step 4: Identify the treated and control groups

Step 5: Split periods into pre and post (labels, dates, or numbers)

Step 6: Analysis frame — optionally collapse to unit x period means

Step 7: The 2x2 cell means and the OLS interaction estimate

Step 8: Effect size in outcome units and as % of the counterfactual

Step 9: Parallel-trends check (needs a time axis + >= 2 pre periods)

Report an Issue