The Devil Is In The Details … And The Data

Posted on May 17, 2018 - By OnPoint Global

Amazon is widely considered one of the most successful e-commerce platforms in the world. The humble online book retailer founded by Jeff Bezos in 1994, now boasts stock prices over $1,500 per share and sells a staggering array of products — from dog food and beauty products, to furniture and industrial tools — in 13 countries. As New Yorker Magazine writer George Packer explains in his op ed on Amazon’s rise to fame: “Before Google, and long before Facebook, Bezos had realized that the greatest value of an online company lay in the consumer data it collected.” And he’s taken that principle all the way to the bank.

Companies like Amazon, that use data to almost eerily predict what their customers need or want before they even seem to know themselves, are living lessons on how to use information to engage their customers, understand their competitors and drive operational success. Big or small, it’s almost impossible to imagine a company that doesn’t value “data-driven decision making” in today’s high-touch, fast-paced digital landscape. In fact, a recent Forrester study found that 44% of B2C marketers are using data to ignite customer action, while another 36% are using data to deepen loyalty and relationships… leaving only 20% of our marketing peers in the dust.

With that said, harnessing data can be more difficult than it seems at first blush. We have a joke in our office that you should never trust data … especially from a data analyst. It’s almost irresistibly tempting to accept data as the unchallenged, gospel truth when it seems to make sense, or when it validates our cognitive biases about what we think should be happening in the marketplace. It goes without saying that this is doubly difficult when the data is beautifully packaged by a skilled data guru who knows how to crunch and fold numbers into elaborate, origami-style insights that elude anyone without a B.A. in data science.

No matter how good the data seems, you should check, double check, and triple check your data. Quite simply: the devil is in the details … and the data. Unchallenged data has a way of lulling you into believing there is only one, optimal way to look at your business because there is a comforting — if not guiling — notion that numbers don’t lie. But this type of thinking can lead to crippling organizational blind spots and wasted time and money. We believe there are three big reasons data analysis fails — but there are plenty of ways keep you out of these pitfalls. So let’s dig in.


Correlation vs. Causality: I can’t count the number of times I’ve heard smart, savvy, successful business people mistake correlation for causality. When one event causes another, this is causality. When two events often occur together, but do not influence each other, this is correlation. Let’s make this concept simpler by using the ever-timeless peanut butter and jelly sandwich as an example. There is a high correlation between people who like peanut butter and people who like jelly because the two ingredients go together like … well … a good PB&J sandwich. The likelihood that someone who likes one half of this dynamic duo also likes the other half is relatively high. But that doesn’t mean that this person likes jelly because he likes peanut butter, or vice versa. His taste reflects a correlation between these two ingredients, not a causality. This example holds generally true. Just because two things are correlated, doesn’t mean that they cause one another. Transitioning to a more business-centric example: media spends do not cause conversion success, but they often correlate.

This is why it’s important to remember that the most actionable customer insights are grounded in a causality, or something that is causing a customer to behave a certain way. When you see data that alludes to a relationship between two events, ask yourself whether it’s a correlation or causality. Then ask someone else to pressure-test your thinking … you might just discover it was the promotion featured in your media spend that caused customers to buy (not the spend itself), which will then give you a deeper understanding of your target audience’s price sensitivity, directional insight into your pricing mix and the perceived value of your product.

Cognitive Bias: A good analyst can support multiple perspectives with her data. A great analyst can sell a decision or a point of view using that same information. No matter what numbers we have in front of us, a living, breathing human is still using her perspective to shade how she interprets and consumes that information, which often includes a reflection of her preexisting assumptions and expectations. Despite the empirical nature of data, there are lots of ways to collect, synthesize, and read — or more accurately: frame — data. And this isn’t unique to digital marketing. One set of numbers can tell dramatically different stories depending on what you do with it in almost any field or industry. You need look no further than the difference between how a company’s financial health is depicted through managerial accounting for internal stakeholders versus financial accounting for shareholders to see this in action on the NASDAQ. When using data, we often search for the answers we believe to be true in our gut. While data can, indeed, validate our hunches, unchallenged data can lead us to believe false truths simply because we want to.

Avoid this blind spot by triangulating your data. This is a trendy data term that simply means using mixed methodology and information sources to validate your results. By triangulating your data, or cross-referencing it through secondary data sets, you begin to challenge and subsequently remove any biases that may exist. You may try pulling your data at different times, exploring proprietary versus public data or comparing conclusions with someone else viewing the raw numbers. Triangulation is most easily conducted quantitatively by manipulating how and where you get your data, but it can also be used to challenge qualitative findings by changing your survey questions or interview style in focus group settings as well.

Garbage In, Garbage Out: Lastly, one of the quickest ways is to botch your analysis is to collect data incorrectly in the first place. The old saying is true: garbage in, garbage out. This is one area where the quality of your data input has a direct causality (see what we just did there?) on the quality of your output analysis. Gathering good data doesn’t happen overnight. You need clear goals and KPIs to know what to track, structured databases that allow you to collate the right information, consistent tagging methodology and a business-wide commitment to uphold these standards. In addition to fueling strategic insights, good data helps your business build relationships. But bad data can lead your decisions astray or derail your customer experience. We all know what it’s like to get multiple emails from your favorite clothing brand calling you by the wrong name or gender simply because their data and CRM platforms aren’t up to date. Data programs are easy to start, but hard to maintain … and the results can impact all aspects of your business.

The trick is to start small. Pick only a handful of variables you’d like to track and the business insights you want to uncover with those pieces of information. It’s okay to go for the low hanging fruit as you build analytics into your sales funnel. Rigorously check your data, recheck it and put it to use immediately. Never collect data for the sake of data. If it isn’t drawing actionable feedback, stop tracking it. Immediately. As your partners and stakeholders see the benefits of how to use the data you’re collecting, it will be easier to enforce quality standards and raise awareness to help challenge data biases and blind spots.

The moral of the story is simple: data is a powerful tool that can very literally transform your business, but you need to check your work, challenge your assumptions and then check and challenge again. Start small, or enlist the help of a strong data-focused agency partner to turn numbers into actions.