You’re not optimizing. You’re guessing…
It tracks everything from test variants to exposures, but shows you surprisingly little.
If all you’re getting are short-term “wins,” here’s what you’re missing:
1. Long-Term Goal Attribution
The UI shows which variant won in 5 minutes. But what about:
- Repeat purchases 30 days later?
- Higher LTV from a specific layout?
- Email open rate 3 months later?
Export Optimizely’s raw event and user exposure data.
Use just that to build multi-session funnels, retention curves, or post-test behaviors.
2. Engagement Fatigue
Ever wonder when a variant stopped working?
Build decay curves from timestamped events. Then spot when users disengage.
3. Internal Rollout Coverage
If you test internally, Optimizely won’t show who never saw it.
Rollouts ≠ exposure.
Better use user_attributes to check who got left out.
Bonus: Combine with Shopify, Meta Ads, or Klaviyo for:
- Real revenue by variant
- Connect ad clicks to test variants to purchases.
- Segment-based variant reactions
Still making test decisions from surface stats?
We turn experiments into data strategy.
About the author. Corinna Zennig leads LazyAnalyst, an e-commerce analytics consultancy: ex-Deloitte, 10 years in business and 35+ projects delivered. More about Corinna