Poor Deming never stood a chance

Source: Hacker News

This post is an elaboration of a shorter post I wrote about five years ago.

Introduction

The two management giants of the mid‑twentieth century were Peter Drucker and W. Edwards Deming. Ironically, while Drucker hailed from Austria‑Hungary (like me, he emigrated to the U.S. as an adult) and Deming was born in the U.S., it was Drucker who proved to be more influential in America. Deming’s influence was much greater in Japan than it ever was in the U.S.

If you’ve ever been at an organization that uses OKRs, then you have worked in the shadow of Drucker’s legacy. You can tell a story about how Deming influenced Toyota, and how Toyota inspired the lean movement, but I would still describe management in the U.S. as Deming in exile. Deming explicitly stated that “management by objectives isn’t leadership,” and I think you’d be hard‑pressed to find managers in American companies who would agree with that sentiment.

I think Deming’s Out of the Crisis is a better book, but I don’t have a physical copy of it.

Why Drucker’s Ideas Stuck More Than Deming’s in the U.S.

It all comes down to the nature of organizations and people.

My rendering of an organization. Not to scale.

An organization is a big, hairy, complex mess, and the bigger the organization, the hairier and more complex it gets. Managers, on the other hand, have a very finite amount of bandwidth. There are only so many hours in a day, and that number does not increase with the complexity of an organization. And, let’s face it, they’re spending something close to 100 % of that bandwidth attending meetings.

How is a manager to make sense of this mess?

OKRs as a Mess‑Reduction Mechanism

In the Druckerian approach to OKRs you:

1. Set a small number of objectives.
2. Identify quantifiable key results for each objective that signal whether progress is being made.
3. Monitor those key results.

Key results reduce the bandwidth required to understand what’s happening in the system. Instead of being overwhelmed by the “blooming, buzzing confusion” of the entire organization, monitoring a handful of key results lets a bandwidth‑limited manager filter out unnecessary detail and focus on what truly matters.

It’s no coincidence that when John Doerr wrote a book on his experience with OKRs at Intel—and how he brought them to Google—he titled it “Measure What Matters.”
Measure What Matters (John Doerr)

The beauty of a set of key results is that they take the messiness of the system as input and produce a neat, actionable summary as output—typically a simple spreadsheet or slide deck.

Deming’s Critique

Deming’s approach to the problem of management was radically different from Drucker’s. In his book Out of the Crisis (see the link), Deming puts his criticism of the “Drucker‑ish” approach in stark terms. He uses the term deadly disease to describe managing through numerical targets.

Eliminate management by objective.
Eliminate management by numbers, numerical goals.
Substitute leadership.

Deming’s perspective can be summed up with the old saying:

If you don’t change the system, the system doesn’t change.

He argued that if you want improvements, you must make systemic changes. Moreover, you have to understand the system before you can devise a system improvement that will actually work.

Key Points

• Focus on the system, not on numbers.
• Leadership replaces target‑driven management.
• Systemic change is prerequisite to real improvement.

Visual Summary

Deming advocated studying the system rather than chasing numerical targets.

Classical Control versus Statistical Control

Deming was not opposed to the idea of goals; in fact, he was a passionate believer that management should strive to improve quality and productivity, and both of those are goals. He was also not opposed to metrics—he advocated applying Walter Shewhart’s statistical techniques to management. It is the use of metrics that differs radically between the two approaches.

Classical (Drucker‑ian) Control

The Drucker‑ian approach is akin to a classical control system, like a thermostat. Specifying the key results is like setting the desired temperature (e.g., 68 °F), and the thermostat generates output to bring the current temperature of the room in line with the set‑point.

The idea is that you give the organization a set‑point, and it implements a control system that works to achieve that set‑point.

Statistical Process Control (Deming)

Deming wrote explicitly about control, but he meant it in a different sense: statistical process control (SPC), which focuses on the variability of the output. I have written about SPC before (see my post on incident data and SPC), but here I’ll revisit the concepts with an example.

Imagine we are observing two brands of thermostats, A and B, each controlling the temperature of a different room. Both thermostats have the same set‑point of 68 °F. We graph the temperature of the two rooms over time:

Note: The points for thermostat A fall within a narrow band, whereas thermostat B shows many more outliers. We would say that thermostat A is under statistical process control, while thermostat B is not, even though the average temperature for thermostat B is closer to the set‑point than that of thermostat A. (In practice you would draw a control chart to determine whether a system is under statistical control.)

Why the Distinction Matters

Deming argued that you must first determine whether your system is under statistical control before deciding what intervention to apply:

In short, with a classical control system you build a mechanism that forces the output toward a set‑point. With statistical process control you observe the system’s variability to understand its behavior before deciding how to intervene.

Drucker Makes a Manager’s Life Easier, Deming Makes It Harder

Deming faced a similar challenge to Calvin.

One of the virtues of OKRs is that they are straightforward for managers to apply. You set direction by specifying objectives, and you enforce accountability by monitoring key results.

Applying Deming’s approach, on the other hand, requires a much greater commitment of management bandwidth. Drucker offers a control mechanism with a bounded amount of information, whereas Deming requires a never‑ending research program with no upper bound on the kind of information that might be relevant. In fact, the information might even be unobservable.

“The most important figures needed for management of any organization are unknown and unknowable.”
— Lloyd Nelson (quoted approvingly by Deming)

I’m personally in the Deming camp, but I can see why Drucker’s ideas won out. In reliability engineering we talk about “making the right thing easy and the wrong thing hard,” a principle sometimes called The Pit of Success (coding horror article). The rationale is that people will tend to do the easy thing over the hard thing. Managers are people too.

However, sometimes the right thing to do is the harder one, and there’s little we can do about that.