I had difficulty controlling my emotions
In her excellent piece Common Obstactles in DataOps, Sarah Floris defines what it means to have a data-driven culture (emphasis mine):
A data-driven culture is a corporate culture that prioritizes the use of data and evidence in decision-making processes. This means that decisions are based on facts and data rather than intuition or personal experience. In a data-driven culture, data is used to inform strategy, identify opportunities and problems, and measure performance.
I wouldn’t change hardly a word of this, except for two: I think “rather than” should be “as well as”.
Tim Harford’s book The Data Detective: Ten Easy Rules to Make Sense of Statistics covers this point in the opening chapters. Each chapter explains a “rule” or guideline to keep in mind when using data, and he begins not with data but with emotions and personal experience.
The first rule is only tangentially related to Sarah’s point, but it’s worth detailing.
Rule #1: Search Your Feelings
Tim Harford writes that “we should learn to stop and notice our emotional reaction to a claim, rather than accepting or rejecting it because of how it makes us feel.”
Of course the point here is that you need access to data first before you can experience an emotional reaction to it, and an organisation that can’t even see its data has to solve that problem before moving on. But once it does, decision-makers are going to feel something when they see it, and their decisions will be informed as much by those feelings as they will be by the data.
Did the numbers go down? Does that reflect poorly on my team? Then my instinct is to reject the data, or at least to express scepticism towards it. Maybe I’m right to do so, and directing the data analysts to reassess their conclusions might uncover data quality or methodological concerns that were missed.
Did the numbers go up? Does that reflect well on my team? Then my instinct is to embrace what the numbers show, and relate them back to all the great work that’s been happening in my area. I don’t even bother questioning the data, but that doesn’t mean that the same data quality and methodological concerns aren’t present. Why bother checking when things are shown to be going so well?
To illustrate this point Harford writes about Abraham Bredius, a leading scholar of the mysterious Dutch master Johannes Vermeer. Bredius was enjoying his retirement in Monaco when, in 1937, he was approached by a former Dutch MP seeking to raise funds for dissidents in Mussolini’s Italy. To do so, he had something of value packed away in a crate that he wanted to sell but required Bredius’s expert opinion first. I’ll let Harford take it from here (this is taken from his website, but the story is the same in the book):
Inside it was a large canvas, still on its 17th-century wooden stretcher. The picture depicted Christ at Emmaus, when he appeared to some of his disciples after his resurrection, and in the top left-hand corner was the magical signature: IV Meer.
>
>
>
Johannes Vermeer himself! Was it genuine? Only Bredius had the expertise to judge.
>
>
>
The old man was spellbound. He delivered his verdict: “Christ at Emmaus” was not only a Vermeer, it was the Dutch master’s finest work. He penned an article for The Burlington Magazine for Connoisseurs announcing the discovery: “We have here — I am inclined to say — the masterpiece of Johannes Vermeer of Delft. Quite different from all his other paintings and yet every inch a Vermeer.”
>
>
>
He added, “When this masterpiece was shown to me, I had difficulty controlling my emotions.”
>
>
>
That was precisely the problem.
As you can probably guess, the piece was a fake. The story of how the forgery was uncovered is worth reading in full, but the point here is that Bredius saw what he wanted to see. As he freely admits, he had difficulty controlling his emotions.
When you approach a dataset, your emotions play an important part in the analytical process. You can learn a lot based on how you feel, and paying attention to those feelings can make you a better analyst.
Rule #2: Ponder Your Personal Experience
In relation to this rule, Tim Harford writes: “We should look for ways to combine the ‘birds eye’ statistical perspective with the ‘worm’s eye’ view from personal experience.”
This, I think, gets to the heart of why I’m sceptical towards the idea that a data-driven culture is one that should base decisions on facts and data rather than intution and personal experience. In the first place, this assumes that choices are made in the absence of data before the data team gets involved, but is that really true?
Decision-makers use the data that is available to them to inform the choices they make, and that data doesn’t always come in the guise of a spreadsheet or a dashboard: sometimes it’s the knowledge they’ve attained through conversation and observation. These experiences are synthesised through reflection and memory into a heuristic model of the world that guides their thinking when decisions are needed.
This all sounds a lot like what we do when we transform and join raw data into domain-specific models, analyse the results, and communicate the insights we discover. It turns out the Modern Data Stack was there all along right inside our heads.
Before you think I’m getting all highfalutin about this, let me bring this back to the superb resource Tim Harford cites to contextualise his second rule.
You’ve probably heard of Gapminder before, but you may not have heard of its Dollar Street project. Go check it out, and you’ll see what I mean when I say that using personal experience as well as facts and data is much more enriching than using one or the other.
The photographs (and the accompanying short videos that show the families getting ready for the shot) are more revealing than the data points alone: I can understand at a cognitive level that the sleeping conditions of a family on an income of $45/month won’t be as comfortable as a family on an income of $583/month, but I don’t feel anything about that until I see the pictures1. And if the decisions I need to make affect the lives of people on low incomes, shouldn’t I make those decisions with some degree of emotion?
This might make some data practitioners uncomfortable: after all, we’re truth-tellers, aren’t we? But there’s room for emotion and personal experience in our work, maybe even room for being biased. Even if we refuse to acknowledge our own emotional states and personal experiences when working with data we should at least acknowledge that others will react to our data products the only way they can: in a human way.
You can read more about how the income values were calculated here: https://drive.google.com/drive/folders/0B9jWD65HiLUnRm5ZNWlMSU5GNEU?resourcekey=0-4rjWstzby3z96urmt8QgpA