In Data We Trust… But For How Long?

The Covid-19 pandemic is exposing all sorts of revelations about life and work in today’s society. Here’s one that’s becoming clearer by the day:

The world is not as “data driven” as we think it is – not even close.

As a co-founder of a data analytics company, I am fascinated by the way Covid-19 is giving the world an unprecedented window into how modern society uses data. Today, we like to think of ourselves as a highly data-driven society, but that is only true in a vanishingly small subset of situations. Most of the time we flounder at best. 

Blind Faith in Technology 

Over the past few decades, our faith in technology innovation and human ingenuity has skyrocketed. We have long been a society that believes “anything is possible” and the accelerating curve of technological innovation has reinforced this idea over and over again.

A testament to society’s obsession with technology: In the midst of all of the craziness with Covid-19, more than 10 million people tuned in to watch the SpaceX rocket launch on Saturday, May 30, setting a new record.

In fact, we’ve made so much progress that technology can sometimes feel like magic: Type a question into a search bar and answers come back. Press a button on your phone and a car shows up. Strap on a wristwatch and get a live stream of data about your health. 

Making it all possible, of course, is data. Driven by major advances across multiple fields of technology in recent years, our use of data has become so sophisticated that digital experiences can feel outright creepy at times. Who by now hasn’t been spooked by an advertisement showing up in their social feed just moments after talking about a product out in the real world? 

Considering all of these technological advances, everyday technology users would be forgiven for thinking that humanity’s mastery of data is much further along than it truly is. If a shoe company you’ve never heard of can track you down and follow you across the Internet, certainly a government can track the spread of a viral outbreak, right?

Optimizing Digital Experiences vs Combatting a Pandemic

As always, the truth is complicated. One answer is that there’s a huge difference between delivering a highly-personalized digital experience and combatting a global pandemic. Both are centered around data analysis, but they are using vastly different means to achieve vastly different ends. What’s more, our ability to harness data in the first category is light years ahead of the second – an important detail that is not readily apparent to most people outside of the data industry today. 

In the first category, optimization is the name of the game and you don’t need perfect accuracy to generate the outcomes you want – the tolerance for error is high and the cost of making mistakes is low. Put the wrong product offering in front of the wrong person at the wrong time and you lose a sale. No big deal – try again and see what happens next time.

Pandemic response, on the other hand, is another beast entirely. In this case, data analysis is all about research and investigation: tracking the spread of the virus, where it’s taking hold, the trajectory of infection rates and other metrics – literally an infinite number of possible measures. The goals also vary. At the highest level, stopping viral transmission is the primary goal that is shared by all. But what exactly is the best way to do that?

Today, the entire world is watching as governments, businesses, and societies around the world try their best to combat the spread of this virus. A considerable part of that effort depends on the ability of people and organizations to capture, share, analyze, make decisions, and take action with data. 

How’s it going so far? If the latest figures are any indication, not so good…

What’s Going On Here?

It will take several articles to explore this question in full, so for now I will keep the focus narrow and say this: I believe that most people, businesses, and governments truly want to be data driven in their decision making in the pandemic, but it is becoming increasingly clear that many do not have the right data, systems, or mindset to do it. 

As is often the case in data analysis, the problems start with the data collection: To get a complete picture of anything, you need more than a single data point. But which ones are the most relevant? The world is an extremely complex and interconnected place, and sometimes the most random and seemingly inconsequential things are the ones that have the greatest impact. Since you can’t always predict what inputs contribute to a given output – and since collecting all possible data about the world is currently impossible – you have to make choices upfront about what data is likely relevant and worthwhile to examine.

Sometimes, for instance, you know there’s tremendous value in a data set, but it simply doesn’t exist yet (e.g. the percentage of people who wear masks correctly). Other times, you have multiple data sets that are clearly relevant and plausibly within reach, but there is no straightforward way to bring them together (e.g. credit card transactions + location data + active cases). You have to find a common denominator to make it work, which requires altering one or more of the data sets, which in turn chips away at the accuracy and fidelity of the data. This can get problematic quickly. The more data sets you join together in this way, the exponentially less meaningful the data becomes overall. 

If you ask me, this is the single greatest roadblock preventing people from being data driven today. It’s also the reason why I co-founded Incorta nearly seven years ago: If bringing data sets together is how you gain a clearer picture of what’s happening in the world, but you have to sacrifice so much detail in the process, where does that leave you? More often than not, spending tons of money and resources to go nowhere quickly. (Yes, we solved the problem by the way.)

Anecdotes, Intuition, and Politics

At the start of the coronavirus pandemic in the US, charts and data from other countries were our primary sources of information about the virus. Stories about the virus within our communities were not yet coming out of the woodwork, so we looked to the data for answers. 

Fast forward several months and people are sick and tired of sheltering at home. They are desperate to resume their lives. They look around their community and see others going back to their normal routines without falling dead. The charts and data tell a different story, but it doesn’t line up with what people are seeing and hearing on the ground. Frustrated with the charts and data, many stop looking.

World-O-Meters, a leading independent international source for data on the Covid-19 pandemic, hit peak traffic and searches mid-April. Meanwhile, Covid-19 continued spreading.

Google Trends data comparing the volume of two search terms between January 1, 2020 and June 30, 2020: “coronavirus data” and “Korean Baseball”. As a country, it appears we are more interested in Korean baseball than data about the novel coronavirus.

In an article I wrote the other month – What the Covid-19 Charts Won’t Tell You About South Korea – I noted:

At times when people struggle to make sense of unexpected crisis, nothing provides comfort quite like charts and data. We put our faith in data because we feel it gives us insight and control...

But what happens when the data is unreliable and/or the tools for engaging with it are too hard to use? Most people give up. Sometimes it’s just easier to fall back on anecdotes, intuition, and politics – tools that humans have relied upon for centuries to gain comfort and control in times of crisis. 

I fear we are seeing a great deal of this today in the fight against Covid-19 and that it does not bode well for whatever comes next. Here are some recent examples at various levels of society that signal to me that we are still a long way from being truly data driven:


All around the world, governments are stuck between mounting pressure to reopen and the lasting health and safety of their citizens. It’s an extremely complex decision matrix that is washed in shades of gray. But one thing is certainly clear: the US is making big mistakes. We are not taking this as seriously as other countries and the latest numbers reflect that. (It’s no wonder the European Union is barring US travelers.)

The data, however, doesn’t seem to be of much interest to some US government officials. In fact, trust in the accuracy of Covid-19 data has been on the decline in the US for months with questions swirling about shoddy data collection practices and other missteps. Just the other week, the head of the CDC confirmed these suspicions when he revealed that Covid-19 cases may be 10x higher than reported. 

The response from public officials across the nation about the checkered accuracy of Covid-19 data is a mixed bag. In San Francisco, for instance, the Department of Public Health is taking action to improve the fidelity of the data, announcing recently that the city will now test all deceased individuals for Covid-19, among other measures. In Florida, meanwhile, the state’s top data scientist was fired in May for refusing to manipulate data to support the state’s reopening.


As governments around the world continue to monitor the spread of Covid-19 and do what they can to prevent it, the business sector largely remains in turmoil. Without a clear end to the pandemic in sight, companies are left to operate in conditions of extreme uncertainty. Hundreds of thousands of businesses have shuttered and the US unemployment rate has already eclipsed the worst months of the Great Recession, with over 20 million people currently out of work. 

The big question now is whether we are barreling into Great Depression territory. That is to say, are these job losses permanent or just temporary? And if they are temporary, how quickly will the jobs return?

If you ask me, the major uptick we are seeing in digital transformation today is a telling signal that businesses see light at the end of the tunnel. According to a recent survey by Deloitte and Fortune Media, 77% of CEOs report that their company’s digital transformation efforts are accelerating during the pandemic. That suggests companies still see plenty of demand – they just need to find new ways to deliver products and services in order to make people feel comfortable doing business with them again.

That’s not to say it will be easy though. As I wrote about in a recent InformationWeek column, the go-to metrics companies use to run their business are going haywire. It’s getting harder to recognize customers and what they want, supply chains are gumming up, and long-held assumptions are being challenged on every front. 


It’s not just governments and businesses that are struggling to make sense of the data and stay agile in their approach to navigating the pandemic – people at the individual level are having to make tough choices in the face of incomplete and sometimes conflicting data as well.

At the time of this writing, for example, the number of active Covid-19 cases is spiking in California and while the data shows this very clearly, you wouldn’t know it by walking outside. One month ago, practically everyone was wearing masks. Today, it’s the complete opposite. Even despite a new mandate from the governor requiring people to wear masks in public spaces – both indoors and out – hardly anyone seems to bother wearing them properly outside anymore. All the while, parks and other outdoor spaces are more crowded than ever.

In the same way that people tend to silence people who tell them news they don’t want to hear, sometimes people do that with data too. When you have a particular belief and you so badly want it to be true – but the data keeps telling you it’s wrong – there’s a good chance you are going to stop listening to the data at a certain point. You will turn instead to anecdotes – stories from your community. I see this happen all the time. I even catch myself doing it.

And that makes sense. We’re only human after all. Using data to make decisions is still a relatively new skill for humanity. For thousands of years, personal experiences and stories were the only sources of data about how the world works.

It’s easy to assume that being data driven is a binary proposition – that it’s all or nothing: you are either data driven or you are not. But that’s all wrong. In reality, there’s a sliding scale to being data driven. Where you fall on that scale is highly situational and depends just as much on external factors (like the availability of data and its perceived value) as it does inner qualities of the individual him or herself.

The same is broadly true for businesses and entire countries as well, and the Covid-19 pandemic is doing more to shed light on this important truth than any other event in recent history. In my next article, I will explore where different countries fall on this sliding scale at this stage of the pandemic and compare the different approaches to being data driven in more detail.

This article is part of an ongoing series on Data Analysis in the Coronavirus Pandemic. Other articles in this series include:

Sign up for email updates:

Published by Matthew Halliday

Co-founder, EVP of Product Strategy at Incorta. Passion for design, UX, data, stories, and making beautiful things.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: