Test&Target versus Google Website Optimizer.

There’s no excuse nowadays not to be optimising across multiple channels.  There’s heaps of tools available from no cost tools to paid tools and there’s plenty of consultants willing to help you get out there and lead you down a path of optimisation.

Just remember though, 90% of optimisation is done by people, only 10% by the tool.  It’s people that create the optimisation strategy, the ideas to test, the segments, the content, the creative and of course, the insights that come from the results – both good and bad.

So how do you know if free is ok, or whether you should invest in something a little more robust?  It depends how serious you are about optimisation.  It also depends on what analytic tools you run as well.  It depends on what you actually want to achieve.

And there’s quite a few distinct differences between Test&Target and Google Website Optimizer, which should make the decision a clear one for you if you’re serious about the whole thing.

This is just a sample of the differences…as I was writing this, I came to the conclusion that there are in fact too many to list (such as mobile testing, in-flash testing, ad network testing), so I tried to focus on what I think are the key differences – the ones that in my opinion, will make or break your venture into optimisation.

A/B Testing.

Both offer it but with big differences.

Google offers it as a landing page redirect, which also means that you need to create two different versions of a full page and host both pages.  Of course, once the test is complete, you’ll need to do something to redirect the tested page to the correct URL.  There is virtually an unlimited amount of content variations you can create though.

Test&Target offers both Landing Page redirects (similar to Google’s A/B) and an in-page A/B test, again using virtually an unlimited number of content variations.  With a T&T A/B test you don’t need to create new versions of the page to test and don’t have the hassle at the end of the campaign.

MVT Testing.

Both offer it.

With GWO, their MVT test is more similar to T&T’s A/B test in that you create multiple variations of the content and use multiple content areas defined on the page.

T&T’s MVT test has multiple content regions that you vary the content in.  The biggest difference is the testing methodology…see method of testing below.

Landing Page (and content rotation).

GWO uses the A/B version (redirect) test to do this, locking the visitor into a specific piece of content.

T&T takes a different approach, of which there are two flavours.  One is a landing page redirect, where you can test complete pages.  The other is a Landing Page Campaign.  The big difference here is that with a landing page campaign, the visitor is not locked into a single offer displayed – they can experience different offers as their profile changes (see profiles below).  A visitor might see different content during the day than at night because of trends (shopping from work or home).  They might see different offers based on the affiliate they came in from, or from the type of search they conducted.  These are typically used for things like Landing Page Optimization using keyword reinforcement.   With GWO you need multiple pages created to achieve this.  T&T, 1 page, 1 campaign.

Method of testing.

Here’s a big difference.

GWO uses full factorial.

T&T uses Taguchi.

Full factorial means that your test must include every permutation of your variables. For example, if you are testing 4 headlines, 4 hero shots and 4 call to action colors, your full factorial is 4x4x4, or 64 variations. That requires a lot of traffic to be able to complete the test with statistical validity in a reasonable time frame. GWO does not support the Taguchi Method, which allows you to include the most options with the least amount of testing and generates results far quicker, with less traffic, than either full factorial or an A/B test.

Segmentation Targeting.

Here’s a massive one.  And to be honest, I don’t think you should be testing if you can’t segment your tests.  In my opinion, it doesn’t provide you much of an insight.

With GWO it’s all or nothing.  Everyone sees the campaign (depending on the % of traffic you’ve sent to the campaign).

With T&T you can target specific segments of traffic, like first time visitors, or visitors who’ve purchased, or visitors who come from an organic brand search term.  You can target on URL, you can target on page query string, on country, by browser or device, by social network or other referring domain, by campaign, by behaviour, such as visits of more than 10 minutes, or more than 5 pages or, in fact by a whole suite of custom rules that you can create.

Segmentation will be required when you want to gradually “roll out” new tests, like a site redesign to only new visitors at first. Existing visitors have probably learned your existing design, and the new one could actually reduce conversion because it requires new learning on behalf of the user.  Something to really consider.

Targeting Rules.

Again a big difference in capability.

GWO has no targeting – period.

In T&T you can create “targeting expressions” which are JavaScript based rules within the engine to determine whether someone meets the targeting criteria or not.  If they do, either enter them into the campaign, or show them a particular variation.  Really good for testing just those who have viewed a product but not purchased it yet.

User Profile parameters.

Another big difference.

GWO – sorry, doesn’t exist, there’s no targeting capability, so it doesn’t need them.

T&T on the other hand allows you to pass data into the platform through the tags and further enhance the visitor profile stored by the platform so you can target variations of content to the visitor.  For example, these can be used when you want to target content based on a visitors category affinity, or based on their total spend – maybe give them a really special offer.  (Ok this is really behavioural targeting).

Traffic Throttling.

Both offer this, but to varying degrees.

With GWO you select the % of traffic you want to enter into the campaign. GWO will then faithfully serve test pages equally to the percentage of traffic you specify. If you have a strong performing page that you want to make even stronger, you actually risk losing money during the test period if your challenger variations are not strong performers. Sending only a portion of traffic to test variations, for example, can reduce this risk. For example, you may choose to send 70% to your control and 30% to your challenger, or 50% to your control and 25% to your two challengers.

With T&T you can select it at the campaign, the targeted content region, and the content variation.  Lots of flexibility.  And for good reason.

Conversion tracking.

Both offer this, but again, radically different in their capabilities.

With GWO you select the page that the user must visit, combined with their code, signifies a conversion has occurred.  A count of.  Yes, you can add the same code to multiple pages so that you can count a conversion when someone visits any of the pages, providing the code is the same.

With T&T you have multiple options.  It could be a click to the content; it could be a specific page with a tag on it, or one of a myriad of SiteCatalyst success events that you’re already tracking.  Many options enabling deeper insight into conversion tracking.  Couple this segment tracking and conversion success metrics and you have very powerful insights.

Conversion Success.

This is critical in my opinion.

With GWO you only get the conversion tracking above.

With T&T you get to see conversion success points along the way.  For example, you can use success metrics already tracked through SiteCatalyst to monitor such things as leads or sales etc; you can use the targeting expressions or the user profile parameters, or any of the other segmentation based options.  Very handy for figuring out if your campaign lifted conversions but your average order size tanked in the process.

Segment Success.

The whole thing about testing is that we, as visitors, do things differently.  So looking at how different segments respond to content variations is fundamental.

With GWO, there is no option for this.
With T&T you can add virtually an unlimited number or combination of segments to monitor how they’re responding.  Again, not much good to have a winning variation if your repeat visitors suddenly decide to spend much less on average than your new visitors, as a result of the campaign.  Your campaign conversion might have lifted but your product managers will want to know why their bottom line has been decimated.

Multiple Live Campaigns.

Not really.

In GWO you can only have multiple live campaigns if they’re different content areas.  You can’t run different campaigns targeting the same content area at the same time.  That means you can’t support your marketing departments campaigns without multiple pages.

T&T allows you to do this.  Based on the campaign entry rules that you define, you can run multiple campaigns into the same content areas.  Why would you want to do this?  So many reasons.  Many companies run multiple simultaneous campaigns.  If you have a modularized design (most likely) then your homepage is likely to feature certain campaign-based areas.  You need to target those campaigns to different segments who meet certain criteria.  With T&T you can.

Nested content.

I’m not aware that GWO can do this.

T&T can for sure.  You might be wondering why you’d want this.  Probably for the more advanced, but it goes back to the multiple live campaigns, and content re-engagement when combined with landing page optimization.  Gets quite tricky, but definitely possible.

Campaign Preview.

With GWO, sorry that’s a no-go.

With T&T use Onsite to preview what your variations will look like on the page, before you go live.  Allows you to validate your test before you go live.  Pretty handy.

Tag based.

Yes both are tag based.  Both require you to put tag onto the page.

T&T uses a <div> tag coupled with a lightweight piece of JavaScript available to any future test, whereas GWO uses fairly heavy JS, customised for this particular test only.

Reporting.

I’ve saved what I consider to be one of the biggest challenges for the end.

GWO gives you conversion reports.  That’s about it.

T&T gives you a whole suit of reports within the T&T interface.  Plus, if you’ve integrated with SiteCatalyst, then all of your metrics are available to you within SiteCatalyst.

Summary.

Herein lies the key.  If you’re optimising for conversion, then you need to fully understand the impact of your testing.  If you don’t you’re flying blind.  What looks like a good result from a conversion standpoint, may in fact be the worst performing result you could have imagined, because you didn’t segment your results, or you didn’t apply all of your other analytic metrics to the results, such as AOV, or Revenue, or Leads etc.

Optimisation unfortunately should not to be taken lightly.  There are many factors involved in understanding whether something was successful or not – you cannot just take a conversion lift at face value.  You might have impacted something else…or you might have seen a massive lift in something else, in which case, you’re a total hero when you report it up the chain.

If you haven’t bought into a paid tool and you’re just getting your feet wet with testing, there’s probably nothing wrong with getting to grips with some of the concepts using Google Website Optimizer.  But, in doing so, you also need to understand it’s limitations and the limitations it will inherently place on your testing and results.

However, if you’re really serious about using optimisation as part of your overall digital strategy, and understand that in today’s day and age, digital optimisation and behavioural targeting is a fact of life, then you should be investing in paid tools, which have the full capability to support your needs.

Of course, more important than the tools are the brains behind your optimisation program. 90%.