Ad Text A/B Tests: How long should you run them?

When A/B testing ad text variations, it can tempting to leap to conclusions too soon before your ads have built up enough impressions. So how many impressions should you wait for before drawing a line under a test?

I suggest running two identical ad text variations in one of your ad groups to find out for yourself. While you would expect identical ads to experience the same Click-through-rate, this rarely is the case when impression levels are low. However, Click-through-rates will converge towards the same underlying value over time. The more impressions you have, the smaller the margin of error.

Monitor your ad text variations on a regular basis to see how their performance metrics change. Once Click-through-rates for your identical ad text variations are close enough, make a note of the number of impressions you had to wait for. Use this number as your baseline impression target when running future ad text tests.

The screenshot below shows performance metrics for two identical ad text variations. Click-through-rates are very different (at 0.65% and 0.52%) even after 3,403 impressions. In this example, we would need to run the test a little longer until Click-through-rates have converged within 0.05% of one another.

Please note that our client’s name has been masked in the screenshot for confidentiality.