Don’t Dismiss the Base Hits

[Originally published September 3rd, 2008 on, an award-winning, but now defunct, Marketing Optimization blog.]

Readers of my blog, and especially those involved in testing, know that conversion rate optimization is the goal we’re after. It’s a great feeling to know that a test you worked on increased conversion or some other KPI, especially when it’s a “Home Run.” I define Home Runs as triple-digit increases in conversion rate.

But one of the dangers of early testing efforts is the problem that some baseball players have: “swinging for the fences.” (Apologies for the baseball references, but it’s getting to be that time of year.) What if your test, or series of tests, doesn’t appear to raise conversion rate? Do you dismiss the tests as failures because they’re not home runs?

Of course not!

Worst case scenario is that you’ve learned something about executing meaningful tests, and about what does or doesn’t resonate with your customers. But more often than not, you are affecting your website in more subtle ways. Remember that conversion rate is often a blended, averaged, blunt instrument. Especially when it’s averaged across large volumes of organic search traffic, SEM traffic, email house list traffic, different product lines, etc.

Here are some things you can monitor when your tests aren’t having huge impacts on your overall conversion rate:

  1. Micro-conversion rates – If you’re testing product detail page layouts and “Add to Cart” buttons, check if those test variables are having an effect on the micro-conversion rate of adding products to the cart.
  2. Funnel conversion rates – If you’re testing lots of minor copy changes to your shopping cart, check for changes in your funnel conversion rate.
  3. Bounce rates – If you’re testing images, copy, or other changes designed to build up the credibility of your site, watch for changes in bounce rates.
  4. Conversion rate by segment – If a key traffic source’s conversion rate goes up in a test, but is “averaged down” by other less important traffic sources, that may in fact be a successful test.

These types of incremental improvements are test results to get excited about! If your micro-conversion rate increases, and your funnel conversion rate stays the same, that’s still more money in your bank account. If you reduce the bounce rate, you’ve gained the chance to convert that customer later, instead of your competitor.

So don’t dismiss the base hits because you’re disappointed about not hitting a home run (this time.) Take it from a patient analyst who’s favorite baseball player was famous for lots of base hits and not all that many home runs 🙂

9 Reasons Your Conversion Rate Fluctuates

conversion rate trend lineAs a practitioner of Web Analytics and Optimization, I’ve spent a fair amount of the last seven years of my career focused on tracking and improving Conversion Rates.This is a noble pursuit for any business, but intense focus on the conversion rate metric can have negative implications – people in your organization (or your clients) may obsess about changes in conversion rate, pester you about them, and even blame you for them!

To avoid this risk and/or annoyance, it helps to have simple, educational “sound bites” for stakeholders explaining why they are maybe/possibly freaking out for no good reason. This is NOT to say that you shouldn’t diligently investigate what you believe to be causes for concern to your business or client, however. Strike a balance, as in all things.

Let’s say someone in your company comes to you with some data about a change in conversion rate. That is a good thing (coming to you with data), right? So, first off, don’t brush them off. Encourage data-driven behavior, even when it might be off-base!

They’re concerned because in a week-over-week report, conversion rate has dropped by 25%. That’s usually a bad thing, so it’s worth some respectful, diligent investigation. Assuming you’ve looked at various data points, and are of the opinion that it’s not a cause for immediate concern, here’s how you might frame the conversation as it continues.

Hi, [Stakeholder], thanks for bringing this to my attention. I’ve dug into the data, looking at YoY behavior, key segments, etc. and I recommend we take a “wait and see” approach. I’m attaching a trended report, so you can see beyond the week-over-week view.

The Stakeholder will almost inevitably ask some follow-up questions about why conversion rate is down. While you will likely have some data points and explanation of your own, here are 9 of my favorite reminders of why your conversion rate may fluctuate from time to time, instead of the constant “up and to the right” trend that we all strive for: Continue reading

The Very, Very Inspiring “DIKW” Hierarchy

D I K W hierarchy

Image credit: Sean Wood

I want to discuss a framework that’s new to me, but not that new: It’s called the Data > Information > Knowledge > Wisdom Hierarchy, or the “DIKW Pyramid.”

I was turned on to it very recently, but it’s been around since at least 1982. It, or variants of it, have been used in fields as diverse as Information Science, Engineering, and Geography.

Different academics in different fields of study seem to disagree about how useful or smart the framework is, but I’ve been thinking about DIKW in terms of Web Analytics and Optimization (natch). And, it’s working just fine for me! 🙂

This framework helps explain the kinds of work I’ve been focused on for the last 10 years or so in Digital Marketing. Kindly allow me to explain…

Continue reading

Video Podcast on Optimization Testing Hypotheses

marketing optimization podcast series logoHi Readers! I’ve not been posting lately, which is sad. I’ll try to get back on the horse in 2014 😉

What I have been up to is a fun video interview and podcast with Alex Harris of the Marketing Optimization podcast series. In it, we discuss my framework of the 5 ingredients of a world-class testing hypothesis. Better yet, we do live optimization on landing page designs, showing you how to use the framework on your site pages!

Check out the session here »


And, if you’re the podcast-listening sort, make sure to subscribe to the Marketing Optimization podcast on iTunes, which features a new podcast with a digital marketing expert every week!

How to Prioritize Your Optimization Roadmap

changing prioritiesWhile case studies about conversion rate lift and increases in revenue from testing get a majority of the “press” these days, there is something far less sexy and far more important to be thinking about: The order in which you execute your test ideas.

I know, that’s way more boring than, Learn how company X increased their sales by 4,000% with one simple change!!!!! I don’t blame anyone for wanting to share exciting test results, but what if you burn an hour on an webinar and find out you’ve already made the change they’re talking about? Or, you’ve already tested that change and it didn’t make a difference on your marketing?

That’s why we’re going to talk about something “boring” that is guaranteed to help you get results over a longer time frame. Prioritization – the way you apply resources to Optimization work to get the best ROI. Continue reading

The Value Optimization Experts Bring to the Table

As the testing and Conversion Optimization of websites becomes more widely accepted and practiced, the technologies that enable testing (A/B and Multivariate testing software) are getting better and cheaper. There is good competition in the testing tool market, so it’s a great time to be a company looking to start a formal testing and Optimization program.

testing tool landscapeWith all the choices for testing technology, a common mistake is for companies to simply buy/license a testing tool and assume that they can “handle” the needs of testing and be successful.

Many companies that go down this road fail after a short time due to the difficulties of setting up valid tests, getting measurable results, and coming up with new testing ideas. Even companies that are somewhat successful at taking on testing without any outside help tend to “plateau” after 6-12 months and aren’t able to get sustainably positive test results.

While I can understand a CMO’s desire to reap the touted rewards of Optimization “on the cheap,” this post will explain why you need staffed Optimization expertise to be successful in a formal testing program. Continue reading

The 5 Ingredients of a World-Class Marketing Hypothesis

5 ingredients marketing hypothesisI recently had a young Optimizer ask me, “How do you turn analytics data into a hypothesis?” My answer was probably unexpected: “You don’t.” My curt answer was meant to alert this young pup that a single input isn’t enough to form the basis of a good marketing hypothesis.

Today’s post will overview what I believe are the 5 key ingredients of a great marketing hypothesis. Plenty of posts I’ve read have instructed you how to leverage the Scientific Method to write a Conversion Optimization hypothesis. They usually instruct you to make sure it’s provable/disprovable, clearly stated, based on a specific Key Performance Indicator, etc.

This is all good advice, but assuming you know all that, I want to cover the inputs. These inputs, IMO, are the difference between a legitimate hypothesis and a world class hypothesis. Some of these 5 key inputs are probably obvious, but a few may have evaded you. Or, perhaps you thought it was “uncool” to have them as inputs?

Continue reading

Bing’s 3 Testing Tenets & Why They’re Crucial

bing logoI recently read a fascinating paper written by some folks at Microsoft called “Online controlled Experiments at Large Scale.” Skip to the end if you want a link to the paper.

The paper’s topic was how Microsoft has scaled its testing and optimization program on the Bing search engine. It was written with somewhat of an engineering bent, so it wasn’t 100% relevant to my world. The kinds of optimization tests I design and conduct are neither automated nor conducted at such massive scale.

However, the authors laid out 3 “Testing Tenets” that they felt were instrumental to the development (and scaling) of their testing program. These simple rules were fantastic to read and 100% relevant to the world of Marketing Optimization and testing!

I enjoyed them so much that I’m going to summarize them for you, with commentary on why you might want to steal them ;-).

Continue reading