There’s good reason to attempt to create a data-driven company. After all, data is easy to collect in large volumes, and at least most of it can be used to inform decision-making. 

But there’s a certain trickiness to data. Most of us can read numbers, and have some understanding of what they mean. Many modern tools also make processing and propagating it even easier. Unfortunately, this can lead to unearned confidence.  

When Fostering a Data-Driven Culture Goes Wrong 

Being data-driven is a highly complicated process that involves numerous experts, company culture, extensive courses and learning materials, as well as software. Not only is it costly; it requires continuous effort. 

Many business leaders tend to skip the cost and effort involved with various facets of data governance, management, and many other aspects. Instead they simply label the company as data-driven and expect employees to follow suit. Great attention will be paid to numerical presentation and various interpretations of data. 

These employees will often be rewarded with praise, or their instructions may be followed with greater rigor than someone who invests less time in gathering numerical data. 

It would be all well and good if data was never ambiguous. Unfortunately, all data can tell conflicting stories, especially when you start connecting several disparate data sources into a single point of interpretation. 

Many organizations fail to take enough care to ensure workers have learned at least the basic principles of data analysis and interpretation, and every employee is left to their own devices. They have access to as much data as they need, but can only reach a surface-level understanding of it. As a result, they will often arrive at inaccurate conclusions, and decisions based on those conclusions can cause issues down the road. Worse, discovering the source of the mistake will be nigh-on impossible.

Common Interpretation Pitfalls

There are plenty of ways data can be misinterpreted. Unfortunately, without expert knowledge, these misinterpretations will often look just as valid as any other. A great example from recent experience has been the answers provided by various Large Language Models. They all seem to make sense, but some of them are horribly wrong. 

Some of the common mistakes are driven by human psychology; some others are bad applications of mathematics. 

Decontextualization. Every metric and number has an underlying origin and context. Various fluctuations and changes may be caused by a multitude of factors, meaning data context has to be carefully evaluated. 

Confirmation bias. Chief among all psychological misinterpretations, confirmation bias is the tendency to first derive a conclusion and then attempt to find data that supports that conclusion. Confirmation bias is often closely involved with other factors that create misinterpretation (such as decontextualization). 

Improper benchmarks. Some data by itself is difficult to evaluate (e.g., conversion rates) so benchmarks are required to make sense of the information. Setting benchmarks based on outdated information, unreliable sources, or even at random, can create misleading conclusions. 

Ignoring variance, seasonality, or time frames. In a business setting, all data will be affected by variance and numerous other factors that may create sudden increases or drops in various metrics. Too many specialists may attempt to derive conclusions from variance, when what they are looking at is in fact randomness or noise. 

Overgeneralization. Team members should be extremely careful about the width of the conclusions they derive from data. Often, there’s a tendency to attempt to apply insights to areas where they don’t apply. 

Operational vs. strategic metrics. Most teams seek operational metrics first, with strategic ones occupying second place. However, when interpreting data, strategic goals should supersede operational metrics. Team members may be led astray by their internal goals instead of looking at what’s best for the company. 

These are just a few of the common pitfalls, and they create problems on an almost daily basis. At the root, the greatest issue with misinterpretation is that, on the face of it, it can look just as legitimate as proper interpretation of data. 

Slapping a label on your business as being “data-driven,” and pushing employees to get on board without making sure they have the skills they need to truly understand how to work with data, is likely to cause harm at some point. Business executives should look at the process of becoming data-driven not as a fancy add-on, but as a reorganization — of processes, skills and mindset. 

If the required resources are not available, it’s often best to have dedicated experts within the field of data analysis supporting your teams, instead of leaving non-expert employees to figure things out on their own. Politics and psychology will take root instead of insight and interpretation, leading decision-making astray. 

Karolis Toleikis is chief executive officer at IPRoyal.