It’s campaign time again. Not just any old campaign, it’s our main recruitment campaign of the year.
And so I thought I’d share a real humdinger with you. What we normally do is behaviourally target content to users based on their application stage.
Why? Because we know from previous tests that behaviourally targeting content for re-engagement purposes not only lifts our application completion rate, but provides more relevance to the user when they visit our site – instead of just seeing a standard campaign message each time. Given the value of an application, and the lift we see, it’s well worth the effort to go down this path.
But this time, the gates of hell opened and out rode the fifth horseman.
We typically use 4 different stages, with one experience (content) per stage. The challenge we always faced was that for each stage, we always wanted to run A/B tests to see which converted better – but could never figure out how to do it when it’s coupled with a behaviourally targeted campaign.
A few months ago, I worked with one of the Omniture Test & Target genius rock stars, implementation engineer Randall, who worked through a nested mbox example with me, which could be used to nest A/B tests within a single Landing Page campaign. Unfortunately at the time we didn’t get to implement anything (or even try it – but it looked great on paper).
Knowing that our primetime Mid Year recruitment campaign period was nearly on us, I dusted off the email and had a good ol’ read, as I felt it had some opportunities for us in a variety of ways.
And so the madness begins…
Always up for a challenge, I strapped on my Test & Target boots, and suggested that we could perhaps, possibly, maybe-ish try to have a crack at putting together a behaviourally targeted campaign, where at each stage of the process we not only change the offer (easy) but run an A/B test (difficult) to test how well different offers convert people to the next stage.
Be careful what you wish for…not only did they like the idea, they compounded it by throwing in another campaign on top.
Mid Year campaign is normally restricted to Undergrad recruitment. At least, that’s been the (now) historical position. But no, this year, we’re doing simultaneous Undergrad and Postgrad Mid Year campaigns.
Yikes!!! Go big or go home! In for a penny, in for a pound.
So, to summarise, we want a simultaneous 4 stage behaviourally targeted campaign with a series of A/B tests within them. And we need to switch users between campaigns based on whether they meet certain criteria, or whether they’ve viewed specific types of content.
Hmmm…a real humdinger!
Test & Target to the rescue.
It gets a bit tricky to do, so I’ll do my best to explain it.
Before I do though, here’s an illustration of what was actually created. Each of the rectangles with the little red circles is a separate Test & Target campaign. You’ll see there are 11 of them. The final bottom row is content…you’ll see there are 12 of them. And you might also notice a total of 10 nested mboxes!
In a bit more detail, we’ve ended up with:
- 1 primary campaign as a Landing Page campaign that switches between an UG or PG experience, with a default offer for those we know nothing about. The UG or PG experience then calls on (via a nested mbox):
- 1 of the 2 recruitment campaigns (UG or PG) as Landing Page campaigns so we can target one of the four stages of the application, via nested mboxes, which calls on:
- 1 of the 8 A/B..n campaigns to determine the offer displayed
There’s a total of:
- 10 nested mbox offers…
- 12 content offers (the things people actually end up seeing)…
- 5 expression targets…
- And 6 profile targets…
Holy moly, I can hear you say. Why so complicated? It’s complicated because of the need to switch people between the campaigns based on what content and stage they’re at.
So, how does it all work?
The top campaign:
As we want to target initially by user interest (they’ve previously viewed Undergrad or Postgrad content, or they’ve visited an Undergrad or Postgrad course, or they’ve visited the Undergrad microsite or Postgrad site, or they’ve come in from an Undergrad of Postgrad email campaign, or they’ve visited the Figure Out Your Course tool or the Postgrad tool), we need to be able to get them into the relevant Undergrad or Postgrad campaign.
So, we need to use a Landing Page Campaign, which is set to re-evaluate the rules (and hence the offers) each time it displays. This allows a user to see different experiences each time, based on the rules above. It’ll switch a user to the opposite campaign if they go, say, from an Undergrad course to a Postgrad course.
But, we don’t want to show them an offer just yet. We’re only doing this to determine which campaign they need to get into – and change that decision on the fly.
Hello nested mbox
On our home page, we already have an mbox (marketing box) defined, with default content. So we start off by creating a campaign to use that mbox.
The following shows the top level campaign, targeting the default mbox on the page – HP_lowerleft.
Instead of serving content back, we want to serve back a different campaign (Undergrad or Postgrad), so we actually need to serve back a nested mbox as our offer.
A nested mbox is just an mbox served into an existing mbox, which then makes another call to Test & Target to get more content.
So, if I meet the criteria for Undergrad, then the content offer that Test & Target serves back to me contains a new mbox. The content offer looks like:
<div class="mboxDefault">
</div>
<script type="text/javascript">
mboxCreate('UndergradNested');
</script>
I repeat that type of content offer for the Postgrad campaign offer, but call the other mbox ‘PostgradNested’.
From a conversion standpoint, we set the Conversion as an SiteCatalyst:event…meaning that whatever the user does, the campaign thinks they’ve converted, and as they restart with the same experience, it re-evaluates them each time, to determine which offer to show them.
That’s campaign #1.
We’ll call it the Mid Year Parent Campaign – and it’s the one at the very top of the illustration.
Now that I have that, I need to create two sub campaigns; one for Undergrad, the other for Postgrad. These campaigns will serve something back into the UndergradNested mbox, or the PostgradNested mbox.
Main Sub Campaigns
So, as I also want to serve back behaviorally targeted content, based on their stage of application, I need to repeat the above type of scenario, but have the experiences based on the users stage.
Stage of application
Once again, a Landing Page campaign allows a user to see different experiences each time, based on the rules evaluated.
So, we created another Landing Page campaign, with 4 offers, each offer targeting the stage that the user has completed:
- They’ve started an application but haven’t completed it yet
- They’ve qualified for entry, but haven’t started an application yet
- They’re a repeat visitor and haven’t qualified yet
- They’re a new visitor and haven’t qualified yet
But, again, we don’t want to show them an offer just yet – we want to show an A/B test for offers 3 & 4, and different content for offers 1 & 2. So, again, we serve back a nested mbox offer, that aligns with the users stage. In the example below, I’ve shown the nested mbox content for Experience D:
<div class=”mboxDefault”>
</div>
<script type=”text/javascript”>
mboxCreate(‘1stTimeVisNested’);
</script>
A/B..n Sub-Campaigns
Now that I have the Undergrad main behaviorally targeted campaign created, I need to create the 4 sub-campaigns which will serve back real content that the user will see.
So, I create a new A/B..N campaign, target the mbox 1stTimeVisNested created above, and create 2 different experiences for it. That’s the A/B test for the 1st Time Visitors who haven’t qualified.
I repeat that for Repeat Visitors who haven’t qualified, but this time I target the campaign at the mbox RepeatVistNested. This one also has two offers (the A/B test for this campaign). Then I create two more A/B campaigns which will serve back content into the respective mbox defined during the other two stages of the application.
Complicated I know – hope you’re still with me.
Rinse and Repeat
Once all that’s been created, it’s a fairly simple matter to duplicate each campaign for a Postgrad version, modify the offer to create new distinct mboxes for each one, and set up the A/B campaigns, with their respective offers.
In doing so, we now have a total of 11 campaigns and 10 nested mboxes, with 12 content offers.
Now to the targeting of it all.
Targeting Expressions and Profiles
Interestingly, this turned out to be the hardest part.
The very first campaign needs to determine whether to ultimately display Undergrad content or Postgrad content.
As we have OR’s in our rules, we need to use an Expression Target (basically Javascript that sits in the T&T target rules and returns a true or false). We have to use an expression because the rules at the campaign level are all AND-based rule, which don’t work for us.
So, we have two Expression Targets on the main top level campaign; one to determine Postgrad, the other to determine Undergrad.
By way of explanation, the Undergrad target expression basically says:
1) Get the value of the last campaign that the user clicked on (that’s our campaign tags)
2) Return true if the users last:
a) coursecategory was Undergrad, or
b) tool used was Mid Year, or
c) LastMicrosite visited was Undergrad, or
d) campaign tag contained “undgrad”, or
e) the current URL query string contains a special one for us to test it with!
The same is basically true for the Postgrad target, just different values.
These two expression targets are then used in the campaign, highlighted below. Despite the fact that it says “…is present”, what it actually means is “… is true”. Never understood why that is, but there we go.
Then we need to set up our Application Stage targeting – basically telling us if the user has Qualified for Entry and hasn’t started and app, or if they’ve started one but haven’t completed it yet. So we added more targets as follows:
You’ll notice in both that there are a combination of both user and profile based rules. User rules are script based and are set up in the Profiles area of T&T and profile based rules are values passed into T&T from an mbox.
We use a lot of mboxes across the site and pass quite a few profile parameters in through the mboxes – which makes it easier for us to target later. All of our key stages have been pre-tagged with mboxes and various parameters.
The only one to use a user-based profile is our lastcampaign profile. Anytime a visitor comes to our site and the visit contains a campaign tracking code, the code is automatically passed into Test & Target and saved into their user profile, via a script, shown below.
You’ll also notice some of the profile.parameters that have been passed in – note that you can’t edit them.
In testing all of this, I just couldn’t figure out why I always ended up in the repeat visitor campaign for either UG or PG.
The reason was easy to overlook, and worth mentioning:
I’d actually structured the order of offers back to front in the second level campaigns…stupid me, I did know this, I clean forgot when I created the campaigns (Omniture: it would be great though to be able to re-order experiences – but you can’t you have to delete and start again). The reason being, you have to put the highly targeted ones at the top and the broadest match at the bottom, as T&T evaluates in the order that the experiences are displayed. And thanks to Veronica at Omniture for spotting a misplaced apostrophe…they’ll get you every time!
Testing everything.
Yes, it’s a challenge – this many campaigns and offers means lots of new sessions, clearing of cookies, and re-establishing the stage of the user journey, but after a few hours, confirmation was established that everything was as it should be.
The biggest problem was that as we were serving back nested mboxes, we couldn’t preview the content in-situ as you would normally do through On-Site Preview, as it looks for the mbox on the page…and they don’t “exist” yet because they’ve not been served back yet.
But we go there.
Performance?
With all of these nested mboxes flying around, I was concerned by performance. But, I was really happily surprised…it’s virtually undetectable. Using WASP we’re able to see the order that the mboxes are served in and they match up correctly.
In Summary
So, there we have it. Two behaviourally targeted campaigns, with triple backflip A/B twists thrown in. The horse is back in the stable, the gates of hell are securely closed again and Rapture didn’t happen.
Test & Target came to the rescue and allows us to do what we need to do from a campaign standpoint.
I can sleep now – the campaign is live, you can experience the different offers by viewing the homepage, then a Postgrad course, then an Undergrad course, then repeating your visit later, or starting an app and not completing it. If you feel so inclined…
