Categories: Content Experiments :

Will a winner be determined before I stop the experiment?

Showing 1-2 of 2 messages
Will a winner be determined before I stop the experiment? Honn 6/15/12 1:46 AM
I have an experiment where it claims that one page has 96% chance to beat the original, but it still says that a winner cannot be determined yet.
Is this because it never determines a winner while the experiment is still running, or is it because it requires a higher visitor count or probability before it determines a winner?

Or to speak plainly, when should you stop a test? When analytics says it has found a winner or when the chance to beat the original is high enough?
Re: Will a winner be determined before I stop the experiment? TylerKatzenberg 6/15/12 1:05 PM
Google has a little information on this topic here: http://support.google.com/analytics/bin/answer.py?hl=en&answer=1745154

Unless your site gets a ton of traffic and you have seen in past A/B tests that you can get meaningful results sooner, I would recommend sticking to the two week minimum. I've seen cases on my site where a variation had a greater than 90% chance of beating the original after four or five days, only to have that advantage completely wiped out a few days later.