Best Day and Time to Send Marketing Emails: A Comprehensive Technical Analysis

1. Introduction

1.1 The Question

When should marketing email go out? Industry rules of thumb name a day and an hour, but they describe averages across other senders' audiences. Your audience may differ by time zone, work pattern and what the email offers, and the differences between slots are often small compared with the noise in measuring them.

1.2 What This Paper Covers

• Putting an interval on an open rate
• Why the best-looking slot misleads (selection and the winner's curse)
• Sequential methods that learn while sending: Bayesian updating and Thompson sampling
• Practical guidance, including which outcome to optimize

2. Background

2.1 Uncertainty in an Open Rate

An open rate is a proportion from n sends. Its standard error is the square root of p(1 - p) / n, so at a 20% rate 500 sends give a 95% interval of about plus or minus 3.5 points (the standard error is 1.8 points) and 5,000 sends about plus or minus 1.1 points. Differences of a point or two between slots cannot be seen with a few hundred sends each.

2.2 Choosing the Best of Many

When you compare many slots and pick the highest, the pick tends to be one that was lucky as well as good. This selection effect, a form of the winner's curse, overstates the winner's true rate, and the more slots and the smaller the true differences, the larger the overstatement. A comparison across many slots also needs a multiple-comparison view: some slots will look better than the rest by chance alone.

2.3 Which Outcome

Open rates depend on the mail client. Clients that preload images count a message as opened before the recipient sees it, and this shifts the numbers in ways that differ by audience. Clicks, replies or conversions are safer outcomes for comparing send times, with the open rate as a secondary check.

3. Approach

3.1 Test Design

Decide the candidate slots, the outcome and the sends per slot before starting, randomize recipients to slots, and send the slots in the same period so the calendar does not confound the comparison. Compare slots with an overall test (for example a chi-square test across slots) before reading individual differences.

3.2 Bayesian Updating

Model each slot's rate with a Beta distribution. A Beta(a, b) prior updates after x opens in n sends to Beta(a + x, b + n - x). The posterior shows both the estimate and its uncertainty, and a prior that expects slots to be similar shrinks extreme observed rates toward the overall average.

3.3 Thompson Sampling

To learn while sending, draw one sample from each slot's posterior and send in the slot with the highest draw; equivalently, allocate each batch in proportion to each slot's probability of being best. Slots that look clearly worse get fewer sends over time, while uncertain slots keep getting tested. It is one of several bandit methods; it trades clean estimates of every slot for fewer sends in poor slots.

4. A Reproducible Simulation

The simulation has 24 send slots whose true open rates are drawn around 20% with a standard deviation of 1.5 points. Part 1 tests every slot with 500 sends and looks at the best-looking slot. Part 2 spends 12 weeks of 10,000 sends under three strategies: spread evenly, commit after week 1 to the slot that looked best, and Thompson sampling with a weekly update (week 1 uniform, then each slot's Beta posterior approximated by a normal). Results are averaged over 400 simulated senders. Random numbers come only from uniform draws turned into binomial and normal draws with inverse CDFs, so the printed results do not depend on the NumPy version.

import numpy as np
from scipy import stats

# Random numbers come only from rng.random() (uniform draws) turned into binomial / normal draws with scipy's
# inverse CDFs, so the printed results do not depend on the NumPy version's sampler algorithms.
rng = np.random.default_rng(14)
def rbinom(n, p):                      # binomial draws
    return stats.binom.ppf(rng.random(np.shape(n)), n, p).astype(int)
def rnorm(mean, sd, size):             # normal draws
    return mean + sd * stats.norm.ppf(rng.random(size))
def rmultinom(total, probs):           # multinomial draws by successive conditional binomials
    out, left, mass = np.zeros(len(probs), dtype=int), total, 1.0
    for i, pr in enumerate(probs[:-1]):
        out[i] = rbinom(left, min(1.0, pr / mass)) if left > 0 and mass > 0 else 0
        left -= out[i]; mass -= pr
    out[-1] = left
    return out

SLOTS, REPS = 24, 400
res = {"overstate": [], "picked_true_best": [], "chi2_p_lt_05": [], "regret_commit": [], "regret_uniform": [], "regret_thompson": []}

for _ in range(REPS):
    # True open rates differ a little between send hours: about 20% with a standard deviation of 1.5 points
    p = np.clip(rnorm(0.20, 0.015, SLOTS), 0.05, 0.6)
    best = p.max()

    # Part 1: one test of 500 sends per slot
    opens = rbinom(np.full(SLOTS, 500), p)
    rate = opens / 500
    k = rate.argmax()
    res["overstate"].append(rate[k] - p[k])                       # how much the best-looking slot's rate overstates its true rate
    res["picked_true_best"].append(k == p.argmax())
    res["chi2_p_lt_05"].append(stats.chi2_contingency(np.vstack([opens, 500 - opens]))[1] < 0.05)

    # Part 2: 12 weeks of 10,000 sends each; compare strategies by the open rate achieved
    def weekly(choice_probs):
        sends = rmultinom(10_000, choice_probs)
        return rbinom(sends, p), sends
    uniform = np.full(SLOTS, 1 / SLOTS)
    # uniform: spread sends evenly all 12 weeks
    o_u = sum(weekly(uniform)[0].sum() for _ in range(12))
    # commit: week 1 uniform, then send everything in the slot that looked best
    o1, s1 = weekly(uniform); pick = (o1 / np.maximum(s1, 1)).argmax()
    o_c = o1.sum() + 11 * rbinom(10_000, p[pick])
    # Thompson sampling, updating weekly: week 1 is uniform (identical priors); after that each slot's Beta posterior is
    # approximated by a normal, and a week's sends are allocated by how often each slot has the highest draw
    o, s = weekly(uniform); a, b, o_t = 1 + o, 1 + s - o, o.sum()
    for _w in range(11):
        mean, sd = a / (a + b), np.sqrt(a * b / ((a + b) ** 2 * (a + b + 1)))
        draws = mean + sd * stats.norm.ppf(rng.random((1000, SLOTS)))
        alloc = np.bincount(draws.argmax(axis=1), minlength=SLOTS) / 1000
        o, s = weekly(alloc); a += o; b += s - o; o_t += o.sum()
    total = 120_000
    res["regret_uniform"].append(best - o_u / total); res["regret_commit"].append(best - o_c / total); res["regret_thompson"].append(best - o_t / total)

print(f"{SLOTS} send slots, true rates about 20% (sd 1.5 points); averages over {REPS} simulated senders")
print(f"best-looking slot after 500 sends each is the truly best slot: {np.mean(res['picked_true_best']):.0%} of the time")
print(f"its observed open rate overstates its true rate by {np.mean(res['overstate']) * 100:.1f} points on average")
print(f"a chi-square test across the 24 slots flags a difference at the 5% level in {np.mean(res['chi2_p_lt_05']):.0%} of senders")
print("open rate given up compared with always sending in the truly best slot (points), 12 weeks of 10,000 sends:")
for name in ("uniform", "commit", "thompson"):
    print(f"  {name:9s} {np.mean(res['regret_' + name]) * 100:5.2f}")

Output:

24 send slots, true rates about 20% (sd 1.5 points); averages over 400 simulated senders
best-looking slot after 500 sends each is the truly best slot: 27% of the time
its observed open rate overstates its true rate by 2.8 points on average
a chi-square test across the 24 slots flags a difference at the 5% level in 61% of senders
open rate given up compared with always sending in the truly best slot (points), 12 weeks of 10,000 sends:
  uniform    2.94
  commit     1.29
  thompson   0.89

With differences this small the best-looking slot was usually not the best, and its observed rate was too high by 2.8 points, almost twice the 1.5-point standard deviation of the true rates. The chi-square test across slots flagged a difference for 61% of senders, so even detecting that send time matters is not certain at 500 sends per slot. Both learning strategies gave up less open rate than spreading sends evenly, and Thompson sampling gave up the least in this run. The ordering is specific to this setup: larger true differences, more or fewer slots, or audiences that change over time give different numbers.

5. Practical Guidance

6. Conclusion

Send-time optimization is a statistics problem before it is a marketing one: many slots, small differences and finite samples. In the simulation here the best-looking slot was usually wrong and too good to be true, and strategies that kept learning did better than fixed schedules. Test with enough sends, put intervals on rates, confirm the leader, and choose an outcome you can trust.