How to Make Testing Successful

One of the questions I get asked most often is, “How can I bring testing and optimization into my company successfully?” I’ve been lucky enough to work with several companies in both the beginning and continuing phases of site optimization so I wanted to lay out some tips that I’ve seen work across most or all of them.

1. Do It Right the First Time

I often have clients who are starting out ask if they can take shortcuts in implementation. The reasons are usually good – “IT won’t let me put the tag on the page”, “I don’t have creative resources right now”, “Upper management is coming down hard on me”, “We’ll take care of it in the next release.” My answer is almost always “No!” Shortcuts both encourage bad behavior and have the potential to hurt you in the long run. For example, let’s say that you as the digital marketer in charge of testing don’t put the tag on the conversion page. Instead, you want to track data through your data warehouse or import data back into the system. You most likely lose the advantage of real-time reporting. Well, what if you have a poorly-performing test but you’re unable to see that until a week later? Or what if you have a test with 85% lift in revenue per visitor but you weren’t able to push those winnings out to your entire population because you also couldn’t see the results until two weeks later? And what if you then try to “socialize” those great results to your group, and people don’t trust the data? As you might have guessed, I’ve seen all these what-ifs happen in the past. The end result is that you have to go back and do it right the second time, but it’s twice as painful and you have to ask yourself what you gained by trying to take the shortcut.

2. Go All the Way

I think that there’s a lot of fear associated with taking the leap from going with your gut to making real data-driven decisions. While we all like to gripe about the HiPPO (Highest Paid Person’s Opinion), I think there is some truth in the HiPPO being your safety net. He or she makes the decisions and you reap the rewards or consequences however it plays out. With testing, you now have to remove the safety net, but you also get to try out some leaps and tumbles. That’s liberating but scary! I’ve seen some companies fail because they try to limit risk by treating testing as project-based task that just needs to be executed once per quarter. By doing so, they actually increase risk because the chances of testing being looked upon favorably decrease with every test that doesn’t generate lift. By limiting the number of tests you run, you limit the number of tries you get to knock a test out of the ballpark.

3. Be Political

We all like to think that our companies are free of politics and drama. However the reality can often be very different, and testing can bubble that up to the surface. What does that mean? Well, there are a lot of people who have something at stake on your site. In fact, it’s really everybody’s site. Let’s take the home page as an example. Merchandisers want to make sure their content makes it. IT doesn’t want page performance or functionality to be affected. Designers want to make sure brand isn’t diminished. So does it seem like the home page is a good place to run your first test? Hmm…probably not. Instead, take 3 things into consideration when trying to find areas to start testing: 1) Is it technically easy to implement? 2) How big is the potential ROI? 3) Perhaps most importantly, is it politically easy to implement? Once you get your testing legs under you and some successes to talk about, that’s when you get to start playing with those pages that might be more politically sensitive.

4. Get Excited

The most successful companies are the ones who get excited about optimization. They showcase and reward tests that generate ROI, they even make fun of and reward the ones that don’t. For example, one company I worked with gave out skunk toys to the marketer who came up with the worst-performing test of the week. I love the idea of still throwing out incentives for those willing to take a risk and put their ideas out there. Of course, you don’t want to find yourself with too many skunks lined up on your desk either! I’m always awed when I see clients reach the point where they can not only present all of the successes and failures of the past quarter, but also speak toward what they plan to test over the next quarter as well. It’s like a testing university graduation where you’ve officially brought optimization into your company successfully.

I’d like to follow up in the future with specific stories from the field. Leave a comment below or shoot me an email at lchiu at omniture.com if you have any tales to share!

• Lily Chiu
• 2 responses
•  Testing and Targeting

• Share on   Digg Reddit Facebook MySpace StumbleUpon Propeller Slashdot Newsvine ma.gnolia
• Bookmark at   del.icio.us Technorati Google Bookmarks Yahoo Bookmarks Yahoo! My Web Windows Live Furl