Adobe Insider and EXBE

The first Adobe Insider with Adobe Target took place on June 1st in Atlanta, Georgia.  I wrote a blog post a couple of weeks back about the multi-city event but after attending the first one, I thought I would share some takeaways.  

The event was very worthwhile and everyone that I talked to was glad to have attended.  The location was an old theatre and Hamilton was even set to run in that building later that evening.  Had I known that my flight back to Chicago that evening would be delayed by four hours, I would have tried to score a ticket.  The Insider Tour is broken down into two tracks.  An Analytics one and an Adobe Target or Personalization one.  My guess is that there were about 150 to 180 attendees which made for a more social and intimate gathering.

The Personalization track got to hang directly with the Target Product Team and hear some presentations on what they are working on, what is set to be released, and they even got to give some feedback as to product direction and focus.

The roundtable discussions went really well with lots of interaction and feedback.  I especially found it interesting to see the company to company conversations taking place.  The roundtable that I was at had really advanced users of Adobe Target as well as brand new users which allowed newbies to get advice and tips directly from other organizations vs. vendors or consultants.

As for the what the attendees liked the most, they seem to really enjoy meeting and working directly with the Product Team members but the biggest and most popular thing for the day was EXBE.   EXBE represents “Experiences Business Experience Excellence”.  You are not alone if that doesn’t roll off the tongue nicely.  Essentially, this all translates to someone (not Adobe and not a Consultant) sharing a case study of a test that they ran.  The test could be simple or the test could be very complex, it doesn’t matter.  The presenter would simply share any background, test design, setup, and any results that they could share.

Home Depot shared a case study at this year’s event and it was a big hit.  Priyanka, from Home Depot, walked attendees through a test that made a very substantial impact into Home Depot’s business.  Attendees asked a ton of questions about the test and the conversation even turned into a geek out.  Priyanka made really cool use of using multiple locations within a single experience.   This capability mapped back to using multiple mboxes in the same experience.  Some advanced users didn’t know it was possible.

So, if you are in LOS ANGELES, CHICAGO, NEW YORK, or DALLAS and plan on attending the Insider Tour, I strongly encourage to submit a test and present it.  Even if the test may seem very straightforward or not that exciting, there will be attendees that will benefit substantially.  The presentation could be 5 minutes or 30 minutes, and there is no need to worry if you can’t share actual results.  It is also a great opportunity to present to your peers and in front of a very friendly audience.  You can register here or via the very nerdy non-mboxy CTA below (see if you can figure out what I am doing here) if you are interested.

Sample Test and feedback…

At the event that day, an attendee was telling me that they don’t do anything fancy with their tests otherwise they would have submitted something and gotten the experience presenting to fellow testers.  I explained that I don’t think that matters as long as the test is valuable to your or to your business.  I then explained a very simple test that I am running on the Demystified site that some might think is simple but would a good example of a test to present.  

Also, at the event, a few people asked that I write more about test setup and some of the ways I approach test setup within Target.  So, I thought I would walk through the above mention Geo Targeted test that I have running on the Demystified website.

Test Design and Execution

Hypothesis

Adam and I are joining Adobe on the Adobe Insider Tour in Atlanta, Los Angeles, Chicago, New York and in Dallas.  We hypothesize that geo-targeting a banner to those five cities encouraging attendance will increase clicks on the hero compared to the rotating carousel that is hard-coded into the site.  We also hope that in the event that some of our customers or previous customers didn’t know about the Insider event, that maybe the test might make them aware of it and they attend.  

Built into Adobe Target is geo-targeting based on reverse IP lookup.  Target user the same provider that is in Analytics and users can target based on zip code, city, state, DMA, and country.  I chose to use DMA so as to get the biggest reach.

This data in this box represents the geo attributes for YOU, based on your IP address.  I am pumping this in via a test on this page.

Test Design

So as to make sure we have a control group and to make sure we get our message out to as many people as possible, we went with a 90/10 split.  Of course, this is not ideal for sample sizes calculations, etc… but that is a whole other subject.  This is more about the tactical steps or a geo-targeted test.

Experience A:  10% holdout group to serve as my baseline (all five cities will be represented here)

Experience B:  Atlanta 

Experience C:  Los Angeles

Experience D:  Chicago

Experience E:  New York

Experience F:  Dallas

I also used an Experience Targeted test in the event that someone got into the test and happen to travel to another city that was part of our test.  The Experience Targeted test enables their offer to change to the corresponding test-Experience.

The banner would look like this (I live in Chicago DMA so I am getting this banner:).  When I go to Los Angeles next week, I will get the one for Los Angeles.  If I used an A/B test, I would continue to get Chicago since that is where I was first assigned.

Profile to make this happen

To have my 10% group, I have to use Target profiles.  There is no way to use % allocation coupled with visitor attributes like DMA so profiles are the way to go.  I’ve long argued that the most powerful part of the Adobe Target platform is the ability to profile visitors client side or server side.  For this use case, we are going to use the server side scripts to get our 10% control group.  Below is my script and you are welcome to copy it into your account.  Just be sure to name it “random_10_group”.

This script randomly generates a number and based off of that number, puts visitors into 1 of 10 groups.  Each group or set of groups can be used for targeting.  You can also force yourself into a group by appending the URL parameter ‘testgroup’ = the number of the group that you want.  For example, http://analyticsdemystified.com/?testgroup=4 would put me in the group4 for this profile.  Helpful when debugging or QA’ing tests that make use of this.

These groups are mutually exclusive as well so if your company wants to incorporate test swimlanes, this script will be helpful.

if (!user.get('random_10_group')) {
var ran_number = Math.floor(Math.random() * 99),
query = (page.query || '').toLowerCase();
query = query.indexOf('testgroup=') > -1 ? query.substring(query.indexOf('testgroup=') + 10) : '';
if (query.charAt(0) == '1') {
return 'group1';
} else if (query.charAt(0) == '2') {
return 'group2';
} else if (query.charAt(0) == '3') {
return 'group3';
} else if (query.charAt(0) == '4') {
return 'group4';
} else if (query.charAt(0) == '5') {
return 'group5';
} else if (query.charAt(0) == '6') {
return 'group6';
} else if (query.charAt(0) == '7') {
return 'group7';
} else if (query.charAt(0) == '8') {
return 'group5';
} else if (query.charAt(0) == '9') {
return 'group6';
} else if (query.charAt(0) == '10') {
return 'group10';
} else if (ran_number <= 9) {
return 'group1';
} else if (ran_number <= 19) {
return 'group2';
} else if (ran_number <= 29) {
return 'group3';
} else if (ran_number <= 39) {
return 'group4';
} else if (ran_number <= 49) {
return 'group5';
} else if (ran_number <= 59) {
return 'group6';
} else if (ran_number <= 69) {
return 'group7';
} else if (ran_number <= 79) {
return 'group8';
} else if (ran_number <= 89) {
return 'group9';
} else if (ran_number <= 99) {
return 'group10';
} else {
return 'sorry';
}
}

Audiences

Before I go into setting up the test, I am going to create my Audiences.  If you are going to be using more than a couple of Audiences in your test, I recommend you adopt this process.  Creating Audiences during the test setup can interrupt the flow of things and if you have them already created, it takes no time at all to add them as needed.

Here is my first Audience – it is my 10% control group that was made possible by the above profile parameter and it has all five cities that I am using for this test.  This will be my first Experience in my Experience Targeted Test which is a very important component.  For Experience Targeted Tests, visitors are evaluated for Experiences from top to bottom so had I put my New York Experience first, I would get visitors that should be in my Control group in that Experience.

And here is my New York Audience.  Chicago, Dallas, Atlanta, and Los Angeles are setup the same way.

Offer Code

Here is an example of the code I used for my test. This is the code for the offer that will display for users in Los Angeles.  I could have used VEC to do this test but our carousel is finicky and would have taken too much time to figure out in VEC so I went with FORM based.  I am old school and prefer to use Form vs. VEC.  I do love the easy click tracking as conversions events in VEC and wish they would put that in Form-based testing.  Users should only use VEC if they are using the Visual Composer.  Too often I see users select VEC only to place in custom code.  That adds overhead and is unnecessary.

Success Events

The success event for this test is clicking on the hero CTA which brings you to the Adobe page to register to join the insider event.  This CTA click was tracked via a very cool function that you all will grow to love as you adobe at.js.

The main segments that are important here are for my Control group.  If you recall, I am consolidating all five cities into the Experience A.  To see how any of these cities do in this Experience, I have to define them as a segment when they qualify for the activity.  Target makes this a bit easier now vs. the Classic days as we can repurpose the Audiences that we used in the Experience Targeting.

Also cool now is the ability to add more than one segment at a time!  Classic had this many years back but the feature was taken away.  Having it now leaves organizations with no excuses for not using key segments in your tests!

An important note, you can apply segments on any and all Adobe Target success events used in the test.  For example, if I wanted to segment out visitors that spent over $200 on a revenue success event (or any event other than test entry), I can do that in the “Applied At” dropdown.  Lot of very cool use cases here but for what I need here, I am going to select “Campaign Entry” (although Adobe should change this to Activity entry:) and I will see how all the visitors from each of these cities did for my Control.

Geo-Targeting

To wrap things up here, I am going to share this last little nugget of gold.  Adobe Target allows users to pass an IP address to a special URL parameter and Adobe Target will return the Geo Attribues (City, State, DMA, Country, and Zip) for that IP address.  Very helpful when debugging.  You can see what it would look like below but clicking on this link will do you no good.  Sadly there is a bug with some versions of WordPress that changes the “.” in the URL to an underscore.  That breaks it sadly but this only applies to our site and some other installs of Word Press.

https://analyticsdemystified.com/?mboxOverride.browserIp=161.185.160.93

Happy Testing and hopefully see you at one of the Insider events coming up!