Question
Did the launch move conversion?
Data
458,119 synthetic buy sessions
Candidate
Syed Ahmed Basharat Ali

A headless checkout lifted conversion 6.1 points. The launch dashboard said 13.2.

A synthetic fiat-to-crypto on-ramp, March to September 2026. Four wallet partners turn on an embedded Apple Pay checkout on May 14 (no redirect, no re-authentication), ten more on July 1, ten on August 15, sixteen never. Every headless session is also played out under the old widget with the same random draws, so the true effect is known: 4,578 purchases and $1.04M of volume caused by the launch.

Four readings of the same launch

percentage points of conversion · line = truth
Launch partners, 6 weeks before vs after+13.2
Headless vs widget sessions, after launch+8.8
Diff-in-diff vs not-yet-launched partners+7.0
Same, within returning and new users+6.7
True effect+6.1

The launch landed in a market rally: partners that hadn't launched yet gained 6.3 points over the same four weeks. Headless sessions are also all iOS and more often returning verified users (58% vs 51%), so a session-level comparison credits the checkout with who the users already were. Split by user type, diff-in-diff lands within about a point of the truth on every wave, and returning users gain about four times what new users do.

Partner conversion gaps are mostly the card issuer

before any launch, same widget everywhere

Partner conversion ranged from 21.0% to 31.5%. Issuer approval alone reproduces 75% of that variance. Six of the ten lowest-converting partners are UK-heavy, and only 4 of the bottom 10 are also bottom 10 for reaching the payment step.

Partner regionCards from banks that block cryptoApproval after a cleared checkout
US10%81.2%
EEA12%78.4%
Rest of world20%72.7%
UK38%62.6%

Headless fraud, read too early

Aug-Sep iOS purchases · chargebacks per 1,000
PathSeen by Sep 30Once mature
Headless0.271.71
Widget1.154.62
Widget / headless4.3x2.7x

Chargebacks arrive weeks to months after the purchase (median 45 days here), and most headless volume is recent. Headless is safer, because Apple Pay is, but the early read overstates the gap and understates headless's own rate about sixfold.

What surprised me

I expected re-weighting by user type to fix the pooled diff-in-diff. It barely moved it (7.0 to 6.7). Per wave and per user type the method was already right; the leftover error came from pooling returning and new users, who respond differently to the rally, across windows where their shares were shifting. The fix is to report the effect by user type, not to correct the total harder.

Run it

stdlib only · about 4 seconds
python3 run.py