The Impossible Promise

Source: Dev.to

AI Forgetting: The Core Problem

In December 2024, the European Data Protection Board (EDPB) gathered in Brussels to wrestle with a deceptively simple question: Can artificial intelligence forget? The board’s Opinion 28/2024, released on 18 December, attempted to provide guidance on when AI models could be considered “anonymous” and how personal‑data rights apply to these systems. Beneath the bureaucratic language, however, lies an uncomfortable truth—the architecture of modern AI makes the promise of data deletion fundamentally incompatible with how these systems actually work.

Large language models such as ChatGPT, Claude, and Gemini have been trained on petabytes of human expression scraped from the internet, often without consent. Every tweet, blog post, forum comment, and academic paper became training data for systems that now shape everything from medical diagnoses to hiring decisions. As Seth Neel, Assistant Professor at Harvard Business School and head of the Trustworthy AI Lab, explains:

“Machine unlearning is really about computation more than anything else. It’s about efficiently removing the influence of that data from the model without having to retrain it from scratch.”

Unlike a traditional database where a row can be deleted, AI models encode patterns across billions of parameters, each influenced by millions of data points. Asking an AI to forget specific information is akin to asking a chef to remove the salt from a baked cake—theoretically possible if you start over, practically impossible once it’s done.

Legislative Attempts to Enforce Deletion

California’s Groundbreaking Law

In September 2024, California became the first state to confront this paradox head‑on. Assembly Bill 1008, signed into law by Governor Gavin Newsom on 28 September, expanded the definition of “personal information” under the California Privacy Rights Act (CPRA) to include “abstract digital formats”—model weights, tokens, and other outputs derived from personal data. Effective 1 January 2025, the law grants Californians the right to request deletion of their data even after it has been absorbed into an AI model’s neural pathways.

The legislation is revolutionary on paper: for the first time, a major jurisdiction legally recognised that AI models contain personal information in their very structure, not just in their training datasets. Yet the technical reality remains stubbornly uncooperative. As Ken Ziyu Liu, a PhD student at Stanford who authored Machine Unlearning in 2024, notes in his influential blog post from May 2024:

“Evaluating unlearning on LLMs had been more of an art than science. The key issue has been the desperate lack of datasets and benchmarks for unlearning evaluation.”

The California Privacy Protection Agency, which voted to support the bill, acknowledged these challenges but argued that technical difficulty shouldn’t exempt companies from privacy obligations. Critics point out that requiring companies to retrain massive models after each deletion request could cost millions of dollars and consume enormous computational resources—effectively making compliance economically unfeasible for all but the largest tech giants.

European Perspective

Across the Atlantic, European regulators grapple with similar contradictions. The GDPR’s Article 17, the famous “right to be forgotten,” predates the current AI boom. When it was written, erasure meant something straightforward: find the data, delete it, confirm it’s gone. AI has scrambled these assumptions entirely.

The EDPB’s December 2024 opinion attempted to thread this needle by suggesting that AI models should be assessed for anonymity on a case‑by‑case basis. If a model makes it “very unlikely” to identify individuals or extract their personal data through queries, it might be considered anonymous and thus exempt from deletion requirements. This raises several unanswered questions:

• How unlikely is “very unlikely”?
• Who makes that determination?
• What happens when adversarial attacks can coax models into revealing training data they supposedly don’t “remember”?

Reuben Binns, Associate Professor at Oxford University’s Department of Computer Science and former postdoctoral research fellow in AI at the UK’s Information Commissioner’s Office, has spent years studying these tensions. His research on contextual integrity and data protection reveals a fundamental mismatch between how regulations conceptualise data and how AI systems actually process information.

Meanwhile, the Hamburg Data Protection Authority has taken a controversial stance, maintaining that large language models don’t contain personal data at all and therefore aren’t subject to deletion rights. This position directly contradicts California’s approach and highlights the growing international fragmentation in AI governance.

The Scientific Quest for “Machine Unlearning”

In 2024 alone, researchers published dozens of papers proposing various techniques: gradient‑based methods, data‑attribution algorithms, selective retraining protocols. Eleni Triantafillou, senior research scientist at Google DeepMind and co‑organiser of the first NeurIPS Machine Unlearning Challenge (2023), has been at the forefront of these efforts.

Triantafillou’s 2024 paper “Are we making progress in unlearning?” (co‑authored with Peter Kairouz and Fabian Pedregosa) presents a sobering reality:

• Current unlearning methods often fail to completely remove information.
• They can degrade model performance unpredictably.
• They may leave traces that sophisticated attacks can still exploit.

The authors suggest that true unlearning might require fundamental architectural changes to how we build AI systems.

Foundation Models: A Special Challenge

Foundation models—massive, general‑purpose systems that underpin many downstream applications—pose a particularly tough problem for unlearning. Their sheer scale, the diversity of downstream tasks, and the opaque way they internalise knowledge make selective erasure extremely difficult. Researchers continue to explore hybrid approaches that combine parameter‑efficient fine‑tuning, knowledge‑distillation, and cryptographic auditing to strike a balance between privacy guarantees and model utility.