The better the autopilot the worse the pilot

Source: Hacker News

Automation doesn’t make operators more careful. It makes them forget how to be. The more reliable the system, the less ready the human.

The argument for automation is that it frees up cognitive bandwidth. Fewer routine decisions means more headroom to think carefully about the ones that matter. What actually happens is the opposite: when a system reliably handles a task, the human monitoring it gradually stops monitoring, because nothing ever goes wrong, and sustained attention without feedback is not something brains do voluntarily. Aviation has a name for this: automation-induced complacency, and it shows up in accident reports where pilots failed to notice system failures they would have caught immediately if they had been flying manually. The irony is that the better the automation, the worse the problem: a system that almost never fails produces operators who are almost never ready for the moment it does. The countermeasure is deliberate:

• Identify the critical tasks you have handed to automation.

• Periodically turn them off and practice manually.

• Keep the interval short enough that the skill does not decay.

Not because the machine is unreliable, but because you are, and the only way to stay ready is to keep practicing the skill the machine has been covering for you.

Schedule regular manual practice for any critical task you have automated. The interval should be short enough that the skill does not decay before the next failure.