I recently had a young Optimizer ask me, “How do you turn analytics data into a hypothesis?” My answer was probably unexpected: “You don’t.” My curt answer was meant to alert this young pup that a single input isn’t enough to form the basis of a good marketing hypothesis.

Today’s post will overview what I believe are the 5 key ingredients of a great marketing hypothesis. Plenty of posts I’ve read have instructed you how to leverage the Scientific Method to write a Conversion Optimization hypothesis. They usually instruct you to make sure it’s provable/disprovable, clearly stated, based on a specific Key Performance Indicator, etc.

This is all good advice, but assuming you know all that, I want to cover the inputs. These inputs, IMO, are the difference between a legitimate hypothesis and a world class hypothesis. Some of these 5 key inputs are probably obvious, but a few may have evaded you. Or, perhaps you thought it was “uncool” to have them as inputs?

The first (these are in no particular order) is Qualitative Data. I find this one is often passed over in favor of Quantitative data. I understand why this happens; quantitative data is black-and-white, it looks legitimate, and it underscores that you’re serious about being a data-driven marketer.

However, Qualitative data can be extremely powerful, even though it doesn’t fit neatly into columns and rows! Have you ever witnessed a usability study and seen a target customer unable to go through a crucial step on your website or app? Ever come across a wonderful turn of phrase via a survey, focus group, or social media that sums up your value proposition in plain language? These are examples of the power of qualitative data, and why you should include it as an input in your marketing hypotheses.

The next input is probably the most obvious: Quantitative data. Web analytics reports are often the first data source consulted when doing Optimization work, so I won’t spend much time covering this one. Just remember that in a rapidly changing digital marketing world, you can’t just rely on Web Analytics data for your Quantitative “source of truth.” You must be ready to delve into multiple tools, multiple data sources, and duct tape them together if necessary to get your inspiration. [Credit to Avinash Kaushik and his book Web Analytics 2.0 for this concept]

While it’s dangerous to over-rely on “best practices,” Heuristic data should definitely be an input for developing hypotheses. Heuristics, or rules of thumb, are codified over time as industry experts do their work, publish findings, and start to see patterns of things that work across multiple tests.

The dangers of relying on Heuristics are at least three-fold. First, there is danger when you take heuristics too literally, as in Thall shalt NEVER have a navigation bar at the top of a landing page. That’s simply too broad of a statement to be taken outright.

Another danger is when people implement best practices instead of test them. As my peer Chris Goward wrote, when people want to implement a best practice outright, the proper response is, “You should test that!”

Finally, relying on heuristics can shut down the creative process that is so important in Optimization. If you simply rely on heuristics, even if you test, you will not try the radical ideas that sometimes produce the “home runs” of Optimization, and you certainly will not be innovative in your work.

Intuition is another tricky one. The whole point of Optimization is to support data-driven marketing, right? So, aren’t we trying to get rid of “gut marketing” that relies on intuition? No! There is absolutely room for intuition in Optimization, and the proper place for it is in the development of hypotheses! A hypothesis for a marketing test is an educated and intuitive guess about how to solve a problem with your website.

It’s OK to be intuitive in Optimization, so long as you can remain completely objective when analyzing test results. The danger with being intuitive is when you have a bias towards the version you believe will win, then torture the data to try and force your intuitive favorite to be the winner.

The final crucial input to world class hypothesis development is involving Stakeholders.
The danger for some Optimizers, as they develop their testing skills, is that they get an ego and decide to ignore business stakeholders as an input source.

Sometimes, frankly, we have to allow stakeholders’ bad ideas into test designs. It’s actually a good thing over time when stakeholders see their bad ideas fail time after time, because they eventually learn that maybe their intuition isn’t as “right” as they’ve believed it to be!

But most of the time, stakeholders have fantastic ideas to help us form hypotheses. They understand market differentiation, the target audience, historical performance, and much more. It would be foolish to downplay the expertise that various business stakeholders bring to the discussion. Involve them…even the ideas you believe are poor…in the Optimization Process. In the long run, your Optimization program will be stronger because it’s more well-rounded.

There you have it: 5 essential ingredients to forming great hypotheses. Do you have a sixth you add to the mix? Perhaps tossing the I Ching? 😉