There were many famous duos throughout the twentieth century who had a positive impact on the world.
Lennon and McCartney brought joy through song. Crick and Watson cracked the genetic code. Jobs and Wozniak invented modern computing.
But there was another duo who arguably had as much impact on the world as any of the names I’ve listed above. There’s a good chance you’ve never heard of them.
That duo was Deming and Shewhart.
The birth of a movement
W. Edwards Deming was many things: his Wikipedia page lists “business theorist, composer, economist, industrial engineer, management consultant, statistician, and writer”. For our purposes he was the father of the quality movement, and if you work with data then learning about Deming’s work will give you a completely different perspective on what you do.
But what about the other half of this duo? Walter A. Shewhart’s list of roles isn’t quite as extensive as Deming’s, but he’s no less important: Shewhart was a physicist, statistician, and engineer, and he invented a chart that few data analysts I know have used, or even heard of.
It’s called the XmR chart (a.k.a. the process control chart or schematic control chart), and here’s the story of how it was first introduced to the world when Shewhart worked at Western Electric:
When Dr. Shewhart joined the Inspection Engineering Department at Hawthorne in 1918, industrial quality was limited to inspecting finished products and removing defective items. That all changed in May 1924. Dr. Shewhart's boss, George Edwards, recalled: "Dr. Shewhart prepared a little memorandum only about a page in length. About a third of that page was given over to a simple diagram which we would all recognize today as a schematic control chart. That diagram, and the short text which preceded and followed it, set forth all of the essential principles and considerations which are involved in what we know today as process quality control." Mr. Edwards had observed the birth of the modem scientific study of process control.
Look at those dates again: the XmR chart was invented a hundred years ago!
It was three years after Shewhart’s remarkable presentation to his boss that he was first introduced to Deming via a mutual friend. I don’t think it’s an exaggeration to describe this as one of the most consequential encounters of the twentieth century.
Big Data is dead
Why do I think that Deming and Shewhart are so important? It’s because the body of knowledge they developed has stood the test of time and informed many of the ways in which companies operate today.
To illustrate this point, look no further than the recent shift in the data discourse precipitated by Jordan Tigani and Tristan Handy.
Jordan Tigani was a founding engineer on Google BigQuery, a cloud data warehouse that can process vast sums of data. But he recently declared that Big Data is dead.
Tristan Handy is the founder and CEO of dbt Labs, the company behind one of the leading modern data transformation tools. But he recently declared that the Modern Data Stack (or MDS) is no longer a useful idea.
Here’s my point: all these things amount to in the end are words that were used at a given point in time to describe a particular trend observed in the data profession. It’s hardly surprising, then, that Handy is now observing a shift in the conversation about AI, which will surely give way in time to the next trend.
Don’t get me wrong: Big Data, the MDS, and AI are not mere fads. Each provided and provides something of value, but they’re ultimately rooted in tools and technology - and I think that explains in large part why they quickly become yesterday’s news.
What are we even doing here?
Which brings me back to Deming and Shewhart.
I’ve spent over a decade working in data, but recently I’ve been thinking: have I really made a difference?
A lot of the time I’m wrangling data to report the same bunch of numbers that often tells the same story, because that’s what I’m being asked to do.
But then, at the start of this year I read Cedric Chin’s piece, Becoming Data Driven, From First Principles. If you haven’t read it already, I recommend you do so now. It’s quite lengthy, but it’s ok I’ll wait.
Reading that piece introduced me to Deming and Shewhart, and it was a revelation for me. Here’s why.
For the early part of my data career I focused on visualising data, data storytelling, and information visual design best practice. I was all into Tableau and Makeover Monday and data viz thought leaders.
Then I became interested in working with data using code, and applying software engineering best practice for greater quality and efficiency. I was all into R and Tidy Tuesday and data science thought leaders.
Reading Cedric’s piece crystallised some misgivings I’ve been having for a while now, and may herald a third phase in my career. Unlike the first two phases, it’s far less concerned with tools, or even with data, than it is with knowledge and quality.
Don’t get me wrong, I’m still interested in data visualisation, data engineering, data analysis, and data science. And I’ll never completely abandon my love for the tools (even if I favour the code-based ones over GUIs these days). But I think Cedric is right: most companies aren’t data driven at all, and I think a fetishisation of data and the tools we use to process it aren’t helping us to address this.
Data trends and tools, whether it’s Big Data or the Modern Data Stack (or, dare I say it, AI) will come and go, so we need to get back to basics and focus more on what will always remain true. We need to ask ourselves why we need data in the first place in order to understand how to realise its value.
In other words, we need to get back to Deming and Shewhart.
12 Days to Deming
I’ll admit: I’m a novice in this area. I’m not naive enough to believe that just because I’m a Certified SAFe Practitioner (#humblebrag) that I’m now a Thought Leader.
Cedric Chin’s piece was the catalyst, but I’ll be taking my first steps into this body of knowledge over the coming months courtesy of the Geelong Quality Council, who are kindly hosting the 12 Days to Deming course. I’ll be participating in virtual study groups, reading copious volumes of text, and engaged in active learning throughout.
The entire thing was masterminded by Henry Neave, who worked alongside Deming towards the end of his (Deming’s) life. Coincidentally, Neave was a lecturer at my previous place of work, the University of Nottingham, and he created the 12 Days to Deming course after he retired and made it freely available. You can access the entire thing here if you like.
One of the reasons I enjoy writing this blog is to share what I’m learning with anyone who might be interested. As I progress through 12 Days to Deming over the coming months, I promise I’ll share with you what I discover and how it might benefit data work of all kinds. I’m sure though that this is just the beginning of the journey - after all, even Deming, who lived until 93, continued to adjust his thinking right up until the end of his life.
There’s no end to the learning process, and that’s the joy of it.