Blog

Why Your Marketing Data Analysis May Be Unreliable

Posted by Jenny Montgomery

March 26, 2014 at 9:01 AM

Scenario: A man gets only two hours of sleep before hopping in his car to drive to work. On the way to the office, he rear-ends a car at a red light. The crash occurred because the man was tired – true or false? data-analysis-importance

The answer is neither true nor false. Without knowing the road conditions, the man’s driving history or whether he was surfing Facebook on his smartphone at the time of the crash, you can’t draw a conclusion from the facts presented. Yet in digital marketing, we often draw conclusions based on limited data, and that can be a costly mistake.

Pretty much anyone can learn how to use a tool like Google Analytics. But it takes a good deal of critical thinking to understand data in context, and that’s a skill that can’t be taught in a short webinar. Analyzing data effectively requires you to question your own conclusions and dig deep for variables that could be skewing results.

Fallacies in market research

Digital marketing is a rapidly evolving field, and many of us have grown into the role of analyst or market researcher, picking up new skills along the way. Maybe you have a liberal arts degree and now you’re being asked to make sense of massive amounts of data – if so, now’s the time to pay attention to some of the fallacies that can ruin your research and cost your company big bucks.

In the hypothetical car-crash scenario, inferring that the man’s tiredness caused the accident is known as a post hoc fallacy – that is, seeing a cause-and-effect relationship when the data doesn’t support that conclusion. It’s one of the many fallacies digital marketers may stumble into, in the quest to understand a target audience.

Another mistake is making conclusions about individuals, based on aggregate group data. That’s called an ecological fallacy, and here’s a real-life example of how that happens, from the company blog for Statwing, a data analysis tool:

There is a negative correlation between a state’s per-capita income and the percent of the vote Mitt Romney earned in the 2012 presidential election. So that might seem to indicate that people with low incomes vote Republican, yet individually, people who vote Republican tend to have higher incomes. What the data does not show you is that in traditionally “red” states, low-income voters tend to vote for Republicans, whereas low-income voters in “blue” states vote for Democrats (and even that’s an oversimplification, for the purposes of keeping this blog post succinct).

Digital marketers can learn a lot from studying common research fallacies. Otherwise, we may not even recognize when we’re at risk for making incorrect conclusions – and subsequently focusing on an audience that has no interest in what we’re peddling.

The importance of a critical eye

Data doesn’t tell you the whole story. We’re the people who create the story, by making assumptions about how data points are related. So one of the easiest safeguards against faulty conclusions is to show your findings to someone within your company who has no knowledge of how you gathered your data. Let that person question your methods, your statistical sample size and how variables could be influencing the data; you may realize you’ve overlooked something important during your research.

Image credit: istockphoto

Subscribe By Email