BrievaA/B subject lines: how the split works and what to expect
What happens when A/B subject testing is on: two subjects per draft, 10% samples, a two-hour wait, and the winner to everyone else. Plus why small lists do not split.
The short version
- With A/B on, every draft comes with a second subject line written from a different angle. You see both on the Issues page before approving.
- On approval, subject A goes to a random 10% of active readers and subject B to another 10%.
- Two hours later Brieva counts unique opens per subject and sends the remaining 80% with the winner. A tie goes to A.
- Lists under 30 active readers send normally with subject A; the samples would be too small to mean anything.
- The Issues page shows the test running, then the winner and both open counts once it finishes.
Turning it on
Open the newsletter, then A/B subject lines, tick Test two subject lines on every issue, and save. From the next draft onwards the writer produces two subjects. If a draft was already waiting for review when you switched it on, it has only one subject; use Ask for changes to have it rewritten and the new draft will carry two.
What the two subjects look like
The second subject is not a reworded copy of the first. The writer is asked for a genuinely different hook: if A leads with a number, B might lead with the consequence for the reader. Both follow your subject prefix if you have set one, and both are kept under 78 characters. The Issues page shows subject A as the issue title and subject B underneath as the A/B alternative, so you can read both before you approve.
If the writer cannot produce a meaningfully different second line, or the two come back identical, the issue sends normally with one subject rather than running a pointless test.
What happens when you approve
Brieva shuffles your active readers with a fixed seed, takes the first 10% as sample A and the next 10% as sample B, and sends each sample its subject straight away. Every message is tagged with its variant so opens can be attributed. The issue moves to A/B test running and the Issues page tells you when the decision is due.
If the newsletter has a scheduled dispatch time or you used Approve for a later time, the split happens at that time instead of at approval.
The decision
Two hours after the samples go out, the scheduler counts how many distinct readers in each sample opened the message. The subject with more unique opens wins; if they are equal, A wins. Brieva then sends the remaining readers, everyone not in either sample, with the winning subject, marks the issue sent, and records the result. The Issues page shows which subject won and the two open counts.
Two hours is a compromise between a meaningful read on opens and not delaying the issue by half a day. Most opens on a business newsletter happen in the first couple of hours; the trend rarely reverses after that.
Why small lists do not split
With 30 readers, each sample is 3 people. One extra open decides the test, which is noise, not evidence. Below 30 active readers Brieva skips the split and sends everyone subject A. The threshold is on active readers at send time, so a list that grows past 30 starts testing automatically.
Even at 30 the samples are small; the test becomes genuinely informative in the low hundreds. It costs nothing to leave it on while you grow.
Things to know
Open tracking depends on the reader's mail client loading images. Some clients prefetch images (which counts as an open) and some block them (which never counts). Both subjects are affected equally, so the comparison holds even though absolute numbers are approximate.
The archive copy and the analytics use the winning subject. The subject shown on the Issues page updates once the winner is sent.
A/B subject lines are available on Pro and Scale.
Ready to set it up?