Most A/B tests fail because the research stage was skipped
A conversion test is only as good as the hypothesis behind it. Research — analytics review, session recordings, heatmaps, user interviews and form analytics — should precede any experiment, because the constraint is usually not the element teams instinctively want to test.
R. Nakamura
Head of Conversion
A conversion test is only as good as the hypothesis behind it. Research — analytics review, session recordings, heatmaps, user interviews and form analytics — should precede any experiment, because the constraint is usually not the element teams instinctively want to test.
Why this comes up so often
In Canada, this question tends to surface at the point where a business has outgrown its first marketing setup but has not yet built the measurement discipline to know what to change. The instinct is to add another channel. The more useful move is usually to establish a defensible baseline first.
What good looks like
- Every claim states its timeframe, baseline and attribution method
- Decisions are documented, including the ones that turned out wrong
- Leading indicators are reported between the lagging ones
- Underperformance is raised the month it happens, not at quarter end
Where to start
Document what you currently believe to be true, then check it. In our experience roughly a third of what a business assumes about its own funnel does not survive that exercise — and finding out early is considerably cheaper than finding out after a year of optimising the wrong constraint.
Frequently asked
How do we apply this in our own business?
Start by documenting your current baseline before changing anything. Most of the value in this approach comes from being able to prove what changed and why.
Related in this cluster
Five business processes you can automate with AI this year
Most businesses overcomplicate their first AI automation project.
Brand identity basics for service businesses
Service businesses often invest in a logo and call it branding, then wonder why their marketing still feels inconsistent.
Facebook Ads vs Instagram Ads: where to spend first
Meta lets you run both from one campaign, so this isn't really an either-or decision anymore.
