How to A/B Test SaaS Copy (Even on Low Traffic)

Blog / A/B Testing

How to A/B test SaaS copy (even on low traffic)

Which copy to test first, how much traffic you really need, and what to do when your page can't carry a test.

How to A/B test SaaS copywriting, even on crazy low traffic

The method in eight steps

  1. Check your traffic first. Know your baseline conversion rate and how many visitors the page gets before you plan anything.
  2. Test messages instead of just words. Low traffic means only big differences are detectable. Test a different claim, not a different synonym.
  3. Start with the highest-stakes copy. Your hero message, pricing page framing and CTAs, in that order.
  4. Build every hypothesis from research. Customer interviews, sales calls and reviews, not a brainstorm.
  5. Fix your sample size and your metric before launch. Then don't peek.
  6. Run full weeks. Two at minimum.
  7. Distrust surprising wins. Check the set-up before you celebrate.
  8. No traffic? Validate differently. Message tests with real buyers, ad and email tests, then ship and monitor.

In the typical SaaS team, copy decisions come down to gut feel.

The founder likes one headline, sales swears by another, product wants the feature name in the hero. Someone senior says "I just don't love it", and that's the end of the discussion.

So the copy goes live on whoever argued the hardest. Nobody really knows if it's working, and when conversions dip, the whole debate starts again from scratch.

It's exhausting, and it makes every copy decision feel like a coin toss. But it doesn't have to be that way.

SaaS copy can be every bit as data-led as your product roadmap. You can make decisions backed by evidence, and stop re-litigating your homepage every year. You just need to know how to A/B test your SaaS copy properly.

Let's go.

What is A/B testing SaaS copy?

A B 50 / 50

A/B testing SaaS copy means showing two versions of a message to randomly split groups of visitors, then measuring which version drives more of one business outcome, such as trials, demo requests or paid upgrades.

Version A is your current copy (the control). Version B is the challenger. Everything else on the page stays the same.

It seems simple enough, but in SaaS, three things make it trickier than the textbook version:

  • The conversion you can measure isn't the one you care about. A click or a sign-up is easy to count. Revenue comes weeks later, after activation, a sales call or a trial.
  • Traffic is usually low. B2B pages rarely get the visitor numbers that consumer sites do, and demo-request rates are often in the low single digits.
  • Your visitors aren't one audience. Self-serve users, enterprise buyers, existing customers and job-seekers all land on the same homepage.

A/B testing vs other ways to test copy

MethodWhat it tells youTraffic neededUse it when
A/B test (split test)Which version drives more conversions, and by how muchHighA high-traffic page, one clear metric
Multivariate testWhich combination of several changes performs bestVery highRarely, for most B2B SaaS teams
Split URL testWhich of two whole pages performs betterHighA full page rewrite or redesign
Message test (buyer panel)Whether target buyers understand, believe and care, and whyNone on your siteLow traffic, or before a launch
Preference testWhich of 2–3 options buyers prefer, and whyNone on your siteChoosing between headline or positioning options

An A/B test tells you what happened, but it never tells you why. For the why, you need research, which is where the best test ideas come from anyway.

How much traffic do you need to A/B test copy?

W A I T (no peeking)

It depends on two numbers: how often the page converts today (your baseline) and the smallest lift you want to detect.

At a 3% conversion rate, detecting a 20% relative lift takes about 14,000 visitors per version, or roughly 28,000 in total. Smaller lifts and lower conversion rates need far more.

I ran the numbers for the conversion rates I see most on SaaS sites. This is visitors needed per version:

Baseline conversion rate10% lift20% lift30% lift50% lift
1%e.g. homepage to demo request163,09542,69319,8277,750
2%80,68221,1099,7983,826
3%53,21113,9146,4552,518
5%31,2348,1583,7801,471
10%e.g. pricing page to trial14,7513,8411,774686
Within 2 months 2 to 6 months Over 6 months
Visitors needed per version. Two-sided test, 95% confidence, 80% power: check any row with Evan Miller's sample size calculator. Shading shows how long each test would take on a page with 3,000 visitors a month, split 50/50.

Now do the calculation for your own page. If it gets 3,000 visitors a month and converts at 3%, that 28,000-visitor test runs for more than nine months.

Three things jump out of that table:

  • A 10% lift is almost undetectable on a typical B2B page, and that's already a big win for a copy change.
  • Higher-converting pages are far easier to test. Pricing-to-trial and sign-up forms beat the homepage every time.
  • The lower your traffic, the bigger the difference between your versions has to be. This is the single most useful rule in copy testing.
The lower your traffic, the bigger the difference between your versions has to be.Amélie Pollak

For scale: Ronny Kohavi, who led experimentation at Microsoft and Airbnb, puts the starting point for experimentation at the moment "you have tens of thousands of users".

Quick feasibility check

Monthly visitorsyour page
×
Monthsyou'd wait (two?)
÷ 2 =
Per-version samplefind it in the table

Read across the table to see the smallest lift you could detect. If it's above 30%, test a bigger idea or use one of the low-traffic methods further down.

What SaaS copy should you A/B test first?

H E R O start here

Test the copy that carries your core message on the pages with the most traffic and the clearest conversion.

That means the homepage hero for most SaaS companies, then the pricing page, then CTAs and sign-up microcopy. Onboarding and email come next.

  1. 1

    Hero message

    Test the claim, not the phrasing

  2. 2

    Pricing page framing

    High intent, higher baseline

  3. 3

    CTAs and the words around them

    Commitment level and what happens next

  4. 4

    Proof framing

    Logos, numbers or a named story

  5. 5

    Onboarding and in-app copy

    Measure activation, not clicks

  6. 6

    Lifecycle email

    High volume, fast results

1. Your hero message

The headline and subhead decide who stays. Don't test phrasing here, test the claim:

  • Category claim vs outcome claim
  • Problem-led vs product-led
  • Who it's for (named audience) vs everyone
  • ❌"Streamline your finance workflows" vs "Simplify your finance workflows"
  • ✅"The all-in-one finance platform" vs "Close your month-end in three days"

The first pair is one idea said two ways, the second is two different promises. Only that one has a realistic chance of producing a lift you can detect.

Haven't sorted your positioning yet? Don't test your hero. Rewrite it first. My SaaS homepage copy guide walks through the whole process.

2. Pricing page framing

The pricing page gets much higher commercial intent and a higher baseline, which makes it one of the most testable pages on a SaaS site.

  • Plan names and who each plan is for
  • Value framing per tier (by team size vs by job to be done)
  • Risk reversal: "Cancel anytime", "No card needed", "Switch plans whenever"

3. CTAs and the words around them

Test commitment level and what happens next, not synonyms.

  • ❌"Get started" vs "Start now"
  • ✅"Book a demo" vs "See it with your own data in 20 minutes"

And test the microcopy under the button. It answers the fear the button creates: spam, a pushy sales call, a card you'll forget to cancel.

4. Proof framing

  • Logos vs one specific customer result
  • A number ("400 finance teams") vs a named story
  • Proof placed next to the CTA vs lower down the page

5. Onboarding and in-app copy

In product-led tools, this is often where the volume lives. Test first-run messages, empty states and upgrade prompts, and measure activation, not clicks. More on this in my product activation copywriting tips.

6. Lifecycle email

High volume, fast results. Subject lines, the first line and the single CTA are the easiest place to learn which angle your audience responds to, then carry the winner back to your website.

How do you write a copy hypothesis worth testing?

W H Y B

A good copy hypothesis names the research insight behind the change, the audience, the new message and the metric that will prove it. If you can't say why version B should win, it probably won't.

Here's the template I use, filled in with a worked example:

Because

[what we learned from research]

eight of our last ten won deals mentioned audit prep in the first sales call,

We believe

[this audience] will [do this] if we change [message A] to [message B]

finance leads will book more demos if we change "The all-in-one finance platform" to "Walk into your audit with every number already reconciled".

We'll know

when [primary metric] moves by [the lift from the table] after [number] visitors per version

when demo requests rise by 30% or more after 6,500 visitors per version.

Notice what that does. It forces a real message difference. It sets the lift before you see any data. And it tells you exactly how long to wait.

Before you write a single variant, go digging:

  • Sales-call recordings and win/loss notes
  • Support tickets and churn reasons
  • Your reviews and your competitors' reviews on G2, Capterra and Reddit
  • A short survey: why did you choose us, what nearly stopped you?

Then check every variant against three questions:

  1. Can the reader picture it?
  2. Could it be proven true or false?
  3. Could a competitor say exactly the same thing?

A vague variant can't win, because there's nothing in it to react to.

Where testable messages come from: my value proposition workshop and messaging pyramid produce exactly this kind of proof-backed message.

How to run an A/B test on SaaS copy, step by step

  1. 1

    Pick one page and one primary metric

    Choose the metric closest to revenue that you can measure in a reasonable time: trials started, demos booked, plans upgraded. Clicks are the fallback, not the goal.

  2. 2

    Size the test

    Use your baseline conversion rate and monthly traffic to find the smallest lift you can detect in the table above. If it's unrealistic, change the page, the metric or the idea.

  3. 3

    Write the hypothesis

    Because, we believe, we'll know. Write it down before anyone sees a number.

  4. 4

    Write versions that differ in one idea

    "One variable" means one idea, not one word. A new headline, subhead and CTA that all express the same new claim still count as one idea.

  5. 5

    QA, then launch at 50/50

    Check both versions on mobile and desktop, check tracking fires, and exclude logged-in customers if your tool allows. They're not the audience you're testing.

  6. 6

    Wait for the planned sample, in full weeks

    Two weeks minimum, so weekday and weekend behaviour are both in. No stopping early because B looks good on day four.

  7. 7

    Decide, ship and write it down

    Record the hypothesis, the result and what you learned, win or lose. A test log becomes your messaging research library.

How do you read A/B test results without fooling yourself?

Decide your sample size and primary metric before launch, don't stop early, and treat a surprisingly big win as a reason to check your set-up. You'll see that a lot of your tested ideas don't win, and a big chunk of significant wins aren't real.

Most tests lose (and that's fine)

~20%

of all experiments win for Optimizely's clients, and only about 10% of those tied to revenue.

Optimizely
20 of 250

ideas tested in controlled experiments at Airbnb had a positive impact on key metrics.

Ronny Kohavi
1 in 5

"significant" wins is still wrong when only about 10% of ideas are better (around 22%).

Success rates from Microsoft, Booking.com, Airbnb

Think of a losing test as a cheap way of not shipping a worse page.

Some winners aren't winners

There's an old rule in data analysis called Twyman's law:

"Any figure that looks interesting or different is usually wrong."Twyman's law

If swapping one word lifts conversions by 60%, check the set-up before you open the champagne.

Five rules for reading results

  • Don't peek and stop. Checking results repeatedly and stopping when they look good inflates false positives. Fix the sample size in advance and wait.
  • Check the split. If you set 50/50 and got 53/47, something's broken in the test, not in your copy.
  • Follow the lift downstream. The rehab-centre headline above lifted CTA clicks by over 400%, but form submissions by over 20%. Still a great win. Just a very different one.
  • Watch for the novelty effect. Returning visitors sometimes react to change itself, then settle. Full weeks and a proper sample protect you.
  • Don't treat "flat" as "safe". No significant difference doesn't mean no difference. Ship a flat variant only if you have a separate reason: it's clearer, more accurate or more on-brand.

Can you A/B test tone of voice?

formal casual T O N E

You can, but tone mostly changes how trustworthy and likeable a brand feels, and those effects are small and slow. So a short conversion test will often miss them. Test messages within your voice, and measure tone with buyer feedback rather than a live split test.

What tone changes

Nielsen Norman Group tested four tone dimensions (funny vs serious, formal vs casual, respectful vs irreverent, enthusiastic vs matter-of-fact) across insurance, banking, home security and healthcare pages. What they found:

52%

of how desirable a brand felt was explained by trustworthiness. Friendliness added just 8%.Nielsen Norman Group

  • Tone shifted how friendly a brand felt by 0.3 to 0.7 points on a five-point scale, and how trustworthy by 0.3 to 0.4 points.
  • Casual, conversational and moderately enthusiastic tones did best. A playful tone undermined trust for insurance, and humour risked alienating users of a home security brand.

So tone is a trust lever first and a conversion lever second. And trust builds across every touchpoint, over months: a two-week test on one page is the wrong instrument for it.

How I'd test tone (and how I wouldn't)

  • ❌Brand voice vs "neutral" voice on your homepageTone, message and length all change at once, so you can't tell what caused the result.
  • ✅Two messages in the same voice, making different claimsA clean read on the message, voice intact.
  • ✅Tone tests on high-volume, low-risk copyWhere tone is the only variable: email subject lines, CTA formality, empty states, error messages.
  • ✅Buyer panelsScore clarity, credibility and "does this sound like a company I'd trust?" alongside, or instead of, a live test.

The brand risk in copy A/B testing

Every small winning tweak pulls your copy towards whatever converted this fortnight. Often that's urgency, generic benefits and a slightly louder CTA. Run enough of those tests and your pages drift towards sounding like every competitor who ran the same ones.

Urgency deserves extra care. Pressure copy can win a test and still land you in trouble: in 2019, the UK's Competition and Markets Authority made Booking.com, Expedia, Agoda, Hotels.com, ebookers and trivago commit to stop misleading claims about availability and popularity. I've written more on where persuasion ends in dark patterns in copy.

Guardrails for any testing programme

  • A voice guide that says what can't change in a test
  • One named person who signs off every variant for brand fit
  • A rule that a winning variant still has to be true
  • A quarterly read of the whole site, top to bottom, to see what all the "small" wins have added up to

Voice not written down yet? That's the first job. It's the core of my verbal brand strategy work.

Can you test copy with low traffic?

clear believable A S K

Low traffic is getting more common. When Google shows an AI summary, users click a traditional search result far less often:

No AI summary15%
With an AI summary8%
Share of Google visits where users clicked a traditional search result. Pew Research Center, 2025.

So yes, you will inevitably have to test copy with low traffic. The way to do it is to switch from measuring behaviour on your site to getting reactions from real buyers, and move A/B tests to the channels where you do have volume, such as ads and email. The goal is the same: evidence before you commit, rather than opinions in a meeting.

1

Research, then rewrite

Pages under a few thousand visitors a month

2

Message test with buyers

A verified panel scores clarity and credibility

3

Moderated reactions

10–15 customers, one word per message

4

A/B test the angle in ads

Volume fast, then carry the winner over

5

Email and in-product

Often more volume than your homepage

6

Ship, then monitor

Before and after, noting what else changed

Six ways to test copy without the traffic for an on-site A/B test.

1. Research, then rewrite

This is the answer for most pages under a few thousand visitors a month. Customer interviews, sales calls and reviews usually tell you what's wrong with the copy. You don't always need a test to fix what customers are telling you directly.

2. Test the message with real buyers

Show your copy to a panel of people who match your ideal customer and collect scores and comments on clarity, relevance and credibility. Platforms like Wynter recruit verified B2B panels for this, and point out what a split test can't give you: an A/B test "will only tell you whether B is better than A and by how much".

Watch out: what people say isn't always what they do. Panellists read carefully; real visitors skim. Use a message test to choose what to launch, not to declare a conversion winner.

3. Run moderated reaction sessions

No budget for a panel? Show 10–15 target customers one message at a time on a call and ask them to react with one word from a set list: clear or unclear, relevant or irrelevant, believable or not. It's quick, cheap and ends the "well, I don't like it" debates.

4. A/B test the angle in ads

Paid ads on LinkedIn or Google give you volume fast. Run two headline angles (pain-led vs gain-led, category vs outcome) and carry the winning angle to your landing page. Measure the landing-page conversion, not just the click.

5. A/B test in email and in-product

Onboarding emails and in-app prompts often see more volume than your homepage. They're a great place to learn which promise your audience responds to.

6. Ship, then monitor

Compare before and after, but be honest that this isn't a controlled test. Seasonality, campaigns and pricing changes all muddy the picture. Keep a note of everything else that changed.

Which A/B testing tools work for SaaS copy?

Website and landing pages

  • Wingify (VWO and AB Tasty, merged in January 2026)
  • Optimizely, good for mature programmes. Its AI variation generator suggests five alternative versions of any text element: useful for widening options, but five variants split your traffic five ways.

Inside your CMS or page builder

  • HubSpot A/B tests any published page, on Content Hub Professional or Enterprise.
  • Webflow Optimize runs A/B tests and AI-driven optimisation, and works on other CMSs through a JavaScript snippet.
  • Unbounce offers classic A/B tests plus Smart Traffic, which routes each visitor to the variant they're most likely to convert on.

In-product and data-led

  • PostHog combines experiments, feature flags and product analytics, so you can measure activation, not just clicks.
  • GrowthBook is open source and free to self-host, and plugs into your existing data warehouse.

Do small copy changes ever make a big difference?

They can, when the traffic is huge. Two famous examples:

+12%

revenue at Bing from a small change to its ad headlines, suggested by an employee: worth more than $100 million a year in the US alone.

Harvard Business Review
8.9% vs 7.5%

sign-up rate when Barack Obama's 2008 campaign button said "Learn more" instead of "Sign up".

NPR

Want to test without the stress?

A/B testing is a way to check a message, but it isn't a way to find one. If your page doesn't have the traffic to test, it needs better inputs: sharper positioning, your customers' own words, and a message you'd bet on even without the data. That's the work I do with SaaS teams.

Book a call with meWebsite copywriting →

A/B testing SaaS copy: FAQs

How long should you run an A/B test on copy?

Run it until it reaches the sample size you calculated before launch, and for at least two full weeks so weekday and weekend behaviour are both included. Don't stop early because one version looks ahead. If reaching the sample would take more than two or three months, test a bigger idea or validate the copy another way.

How many visitors do you need for a copy A/B test?

It depends on your conversion rate and the lift you want to detect. At a 3% conversion rate, detecting a 20% lift needs about 14,000 visitors per version. At a 10% conversion rate, the same lift needs about 3,800. Low-converting pages with a few thousand monthly visitors usually can't detect realistic copy wins.

What's the difference between copy testing and A/B testing?

An A/B test splits live traffic between versions and measures which one converts better. Copy testing, often called message testing, shows copy to target buyers and asks how clear, relevant and credible it is. A/B testing tells you what happened. Message testing helps explain why, and works without site traffic.

Should you test one word at a time?

Only if your traffic is enormous. Single-word changes usually produce effects too small to detect on a B2B SaaS page. Test one idea at a time instead: a different promise, audience or proof point, expressed consistently across the headline, subhead and CTA.

Can you A/B test copy with low traffic?

You can rarely run a reliable on-site A/B test, but you can still test. Validate messages with buyer panels or moderated reaction sessions, test headline angles in paid ads or email where volume is higher, then ship the winning message and monitor before-and-after results with care.

Can AI write A/B test variants?

AI is useful for widening the range of angles to consider and for summarising customer research. But more variants split your traffic further, and unedited AI copy tends to drift towards generic phrasing. Use AI for options, then write the finalists yourself, in your brand voice. (Wondering about the SEO side? Here's whether Google can detect AI content.)

Amélie Pollak

Amélie Pollak is a copywriter and verbal brand strategist for SaaS startups and scale-ups, based in Montpellier. She's run 50+ messaging and tone of voice workshops.

LinkedIn · Work with me

Amelie Pollak

Amelie Pollak is a copywriter and brand strategist working with SaaS startups, scale-ups and growing businesses to make their words work harder.

https://ameliepollak.com
Previous
Previous

How to Write Competitor Landing Pages for SaaS

Next
Next

How to Hire a Copywriter for Your Startup's Brand Voice