Data Beats Opinion… or does it?

Data Science is trending! Year by year more and more online businesses start their first data science projects, build up their data teams, hire more data scientists and make their more organization data-driven. That’s awesome! The “data movement” is growing.

But as in every movement, the most excited folks in it tend to make exaggerations and overstatements.

One of the most dangerous of these is: “Data Beats Opinion.”

Data beats opinion?

No, it doesn’t – and it shouldn’t.

In this article, I’ll tell you why!

Note: If you are absolutely new to the topic of business data science, I recommend starting with my comprehensive introduction article: Data Science for Business. It’ll guide you through the important basics.

Data, opinion, gut feeling and more…

One thing I hate about the data beats opinion phenomenon is that it implies that data and human intuitions are the enemies of each other.

They are not. In fact, they should be the closest allies.

I’m a data analyst at heart, I’ve been working on data science projects for many years, and from my personal experience I can tell you two things about data:

1. To start a data project, you need human input.
2. To draw conclusions from a data project, you need human input.

To start a data project, you need human input

Before a data scientist can start to work on a new project, she needs answers for quite a few essential questions:

1. What’s the business goal of the data project?
2. What are the most important questions she wants to answer with data?
3. Will the data project make impact? How?
4. Where should she start?
5. …

These can be partially answered by standard data discovery (or in other words – by playing around with the raw data.)

But if you have ever worked on a real data science project, you know that the easiest and fastest way to get real answers is by actually asking your colleagues. They have the business experience, they’ve talked to real users and customers, they know what worked and what didn’t work so far. And yes, maybe they didn’t do it in the smartest, most data-driven way (yet)…

But their input – their opinion – will give you a reference point when you start your data project.

Your colleagues will tell you their hypotheses and maybe you will prove them wrong with your data project. But a potentially wrong hypothesis is a 1000 times better start for a data scientist than no hypothesis at all.

To draw conclusions from a data project, you need human input

Let’s play a game!
Here’s a chart about the main funnel of a note-taking app:

How would you act on this data?

Would you focus on fixing the conversion between registration and first-note-creation? (First step of the funnel.) Makes sense! People who register but never start to use your app are an easy target audience to help and thus a great way to optimize your funnel.

But wait… wouldn’t it be more logical to help people finish the process and convert the open-note-again users to close-note users? (Last step of the funnel.)

No, no… Shouldn’t retention be your number one priority? In that case, fix the save-first-note to open-note-again ratio first! (Middle of the funnel.)

Are you getting it?

The decision you make based on your data is just another opinion, too.

Of course, it’s a more educated opinion than one without any data. And you can improve it even further by running more analyses (one chart is usually not enough to see the whole picture), by conducting user interviews or by understanding the context better…

But that doesn’t change the fact:

When you create a new analysis and show it to 10 people at your company, they will draw at least 3 different conclusions from it.

And that’s normal.

That’s hard-wired into the DNA of data science projects.

Data is not the single source of truth

Another thing I hate about the data beats opinion expression is that it implies that with data you can learn everything.

That’s not true, either.

Here’s a schematic diagram that helps you to imagine how the discovery process works in real data projects.

The black circle is the unknown area, and the white circles are the things you discover, analysis by analysis…

There will be overlaps between the results of the analyses. And there will be partial truths that you will be pretty certain about. But you will never see the full picture.

As I said, it’s hard-wired into the DNA of data projects.

Why is that?

One question that data will never be able to answer

The result of a data analysis is always an abstraction. The data scientist’s job is to compress thousands (or millions) of user behaviours into a few charts. Of course, it will never show you the full picture.

But data has another limitation, too.

Data can help you to answer many questions:

• What do your users do?
• When are they doing it?
• How are they doing it?
• …

But it will never answer a crucial one:

Why they are doing it?

One more time: Data will never be able to answer the “why” question. And if you think about it, quite often, that would be the real game-changer. Those “why” questions could help you to understand your audience’s reasons, motivations and desires.

Don’t worry!
Even if you will never have a complete, 100% certain and clear answer, fortunately, you don’t have to completely give up on it, either.

Here are few things that can take you closer to answering the why questions:

• Running qualitative research – like usability testing, five second testing or simply talking to your users – will help you to get these in-depth insights from a few users. And that will give you at least an idea of what your whole audience thinks.
• If not a direct answer, a comprehensive research with multiple data analyses will give you indirect answers and strong suspicions. (E.g. Let’s say all of your users use one of your features. Right after that, half of them open a support article that explains how that feature works. That’s a pretty strong signal that 50% of your audience doesn’t get how that feature works… Or, maybe it’s the complete opposite: they are your power users who got excited by that feature and want to learn how your product works, in depth.)
• And of course, the best is if you combine the two: data analysis and qualitative research. With that, you can get in-depth and quantitative insights, too.
  

The point is, data is definitely not the unbeatable single source of truth that data-enthusiastics like to paint it as.

Like every other research method, it has its own limitations.

And that leads to our final conclusion.

Data Beats Opinion? No. Data helps opinion!

As I said, data and opinion are not enemies, they are strong allies.

Data helps opinion.

And opinion helps data.

In my mind a great data-driven organization works like this:

1. Managers, marketers, business people, etc. work closely with data professionals.
2. The business professionals come up with hypotheses, ideas, gut feelings, etc. And when they do, they go to the data scientists and ask them to prove or disprove their ideas with data.
3. The data scientists run comprehensive research to answer these questions.
4. The business people and the data scientists – together – draw conclusions from the results of the research. They prove or disprove the original idea.
5. They also come up with alternative solutions, new ideas and further research plans.
6. If the data actually supports the original idea (proven by data analyses, UX research and A/B tests), the company acts on it and implements the changes.
7. The data scientists measure the long-term impact of the changes and follow up in a few months.
   

If this sounds something you would love at your company, take a closer look at my Data Strategy course:

Conclusion

Data doesn’t beat opinion.

It most definitely does not.

Data helps opinion.

And whether you are a data scientist, a data enthusiast manager, or a data sceptic: you should keep that in mind!

Data is important. Working without data is like driving a car with closed eyes. You will hit the wall strong and fast.

But listening only to data is not much better, either… That will make you a half-blind driver. Still very dangerous – if not even more.

To see clearly:

Listen to gut feelings, listen to intuitions, listen to opinions – and use data to prove, disprove and improve all these!

…but this is only my opinion.

If you disagree, show me the data that disproves it! 😉

• If you want to learn more about how to become a data scientist, take my 50-minute video course: How to Become a Data Scientist. (It’s free!)
• Also check out my 6-week online course: The Junior Data Scientist’s First Month video course.

Cheers,
Tomi Mester

Cheers,
Tomi Mester