Purpose

This scatter plot isolates the relationship between search position and click-through rate to determine whether the treatment's CTR improvements stem from better positioning or from title/content changes that drive clicks independent of position. By overlaying observed data against the expected CTR curve, it reveals whether the treatment variant is outperforming or underperforming industry benchmarks at its achieved positions.

Key Findings

Position Improvement: Treatment variant achieved mean position of 9.53 vs. control's 11.76 (−2.23 position change), moving closer to top results where higher CTR is expected
CTR vs. Expected: Treatment points cluster near or slightly above the expected CTR curve, suggesting performance aligns with position-based benchmarks rather than exceptional title/content quality
Curve Alignment Pattern: Most pages show treatment points tracking parallel to the expected curve, indicating position changes drive CTR gains rather than independent title optimization

Interpretation

The treatment's +0.27x adjusted CTR lift appears largely attributable to improved search positioning rather than superior title or content relevance. With treatment pages ranking ~2.2 positions higher on average, they naturally capture more clicks according to industry expectations. Only the promoted tutorial page (0.72x lift) shows meaningful divergence from the curve, suggesting genuine title/content superiority beyond positional advantage.

Context

Low absolute