Google Content Experiments (GCE) is a somewhat different approach from either standard A/B or multivariate testing. Content Experiments is more A/B/N. You're not testing just two versions of a page as in A/B testing, and you're not testing various combinations of components on a single page as in multivariate testing. Instead, you are testing up to five full versions of a single page, each delivered to visitors from a separate URL.
What you can do?
With Content Experiments, you can:
- Compare how different web pages perform using a random sample of your visitors
- Define what percentage of your visitors are included in the experiment
- Choose which objective you’d like to test
- Get updates by email about how your experiment is doing (not currently available)
Better than Visual Website Optimizer?
Google Content Experiments is probably an improvement from Google Website Optimizer, however if you are wondering if this new approach is better than Visual Website Optimizer, the answer in my opinion is: NO.
“In conclusion, Google Content Experiments is a tool most suitable for those just starting out with A/B testing and conversion rate optimization. It has all that any beginner would require to understand how CRO works and what it can do for an online business".
Pros and Cons of GCE1) Integration with Google Analytics – Google Content Experiments is now a part of Google Analytics and this means better integration between the two.
2) Your experiments will be terminated after 3 months – All content experiments currently have a maximum life of 3 months. If you want to see data after that, you’ll have to create a new experiment and combine the data between the two. Usually, small improvements take a lot of time to achieve statistical significance while large improvements happen fairly quickly. If any of your experiments used to take longer than 3 months, you’ll have to learn to create bigger changes that show bigger results.
3) Experiments have to run for 2 weeks before Content Experiments reports statistical significance – This is a constraint if you have a large number of visitors to your website and statistical confidence can be calculated in a timeframe lesser than 2 weeks. In such a scenario, you could end up losing sales as part of your visitors will be shown a low performing variation for a period of time that is longer than required. However, test results will be shown live.
4) Advanced Segments will be available to slice & dice your data – All Advance Segments are available, meaning that you can now very finely slice & dice your data and draw better insights about your visitors’ behavior. This is a very useful addition to GCE that will enable you to finetune your funnel for different customer segments.
5) You cannot stop low performing variations before the test is over – In a running test, you will sometimes see a variation that is obviously not doing well and turn it off. This is especially so in large eCommerce websites where rogue unstoppable variations could result in a significant loss of revenues. While GCE won’t have a way to stop low performing variations during a running test, it will try to dynamically shift traffic away from them.
6) You can have maximum 5 variations per test and not more than 12 active tests – These shouldn’t be a hindrance for most small businesses but the same cannot be said for larger eCommerce websites. Websites with a lot of traffic benefit greatly from regular testing on different steps of the funnel and limiting the number of tests will prove to be a pain point.
7) You can now use URL variables and Regular Expressions – You can run tests across domains and include / exclude URLs using regular expressions. This will really help to match the tests to a path as your visitor will move from page to page. REGEX and URL variables are also helpful when you want to exclude certain IPs from a test, for example, your own company IPs so that your actions do not count as goals and skew the results.
8) No multivariate testing – This is a very important feature that Google inexplicably decided to leave out. Multivariate testing enables a quicker understanding of which changes work and which don’t. The absence of this feature is a considerable drawback for large eCommerce websites who have the visitor flow to achieve statistical significance quickly.
9) Email notifications are now available – Useful feature because you won’t have to monitor your tests constantly – set up the notifications and let Google tell you what’s happening when.
10) Revenue goals are missing – Another important functionality that Google should have included in GCE is the ability to track eCommerce transactions as goals. Currently, only URL and events based goals can be tracked so you’ll have to use a thank-you page to achieve the desired outcome. It’s not a crippling feature, but is still a “hack”.
11) Different code snippets for different experiments – What they should be aiming for is one single code snippet that allows you to run multiple tests on your page. For example, Visual Website Optimizer requires you to place the code snippet only once on a page and then, any number of tests can be run on the page simultaneously. For GCE, each Experiment has its own code snippet and you can have only one running at a time. This has two implications
- You can’t run more than one test on a step of the funnel meaning it takes longer for you to optimize your website and see the result;
- If you want to run a second test after 2 weeks, you’ll have to dig in and replace the code snippet.
Finally, a video on Google Content Experiments by Google:
0 comments:
Post a Comment