당신의 AI 제공업체가 단일 장애점이다.

출처: Dev.to

Last Friday, the U. S. Commerce Department sent a letter to Anthropic. By that evening, Fable 5 and Mythos 5 were gone. Not deprecated. Not throttled. Gone. API calls returned 404s. Live sessions errored out mid-conversation. Production applications that depended on those models simply stopped working.
Three days after launch. No warning. No migration window.

And honestly? We got lucky this time. Fable 5 was only available for three days. Nobody had time to build real production dependencies on it. Imagine this happening to a model you’ve been using for six months. A model your entire product depends on. That’s the scenario you should be planning for.

I would like to ask you something. If your database vendor could be forced to shut down your primary database with a single government letter, would you run it without a failover? Of course not. But that’s exactly what most teams are doing with their AI provider.

Most teams treat their AI provider like electricity. You flick a switch, the light goes on. You don’t think about where it comes from, you don’t think about what happens when it stops. You just expect it to work. You pick a model, hardcode the API endpoint, build your prompts around its quirks, and ship. It works great. Until it doesn’t.

And look, I get it. When you’re building fast, the last thing you want to think about is “what happens when my model disappears.” But this week proved that’s not a theoretical risk anymore. It’s not even about uptime.

Your model can be pulled for regulatory reasons. For policy changes. For geopolitical drama that has absolutely nothing to do with your application. The Anthropic situation wasn’t a bug. It wasn’t infrastructure failure. It was a regulatory kill switch. And it affected every single customer worldwide.

I’ve written before about the hidden costs of depending too heavily on AI tools without understanding what’s under the hood. This is the same problem, just at a different layer of the stack.

This frustrates me. We already know how to do this. We’ve spent decades building resilient systems. We don’t run a single database without replication. We don’t rely on one CDN. We put load balancers in front of everything. We design for failure because we’ve been burned enough times to know that everything fails eventually.

But somehow, when it comes to the model layer, we forgot all of that.

Teams are building entire products on a single provider’s API with zero fallback. No abstraction layer. No alternative routing. No graceful degradation. Just a direct dependency on one vendor’s model, and a prayer that nothing goes wrong.

That’s not engineering. That’s hope-driven architecture.

So what do you actually do about it? The patterns aren’t new. You just need to apply them to the model layer the same way you apply them everywhere else.

Abstract your model calls behind an interface. Your application shouldn’t know or care which provider is serving the response. When one goes down (or gets shut down by a government letter), you route to another.

This doesn’t mean you need to maintain identical prompts across five providers. It means you design your system so that swapping a provider is a configuration change, not a rewrite. And yes, there’s a cost. Maintaining that abstraction layer is real engineering work. You’re building and testing against multiple providers, handling different response formats, managing prompt variations. It’s not free. But neither is waking up on a Saturday morning to find your only provider is gone and you have no plan B.

If you run the model yourself, nobody can switch it off remotely. Full stop.

Open-weight models give you that. They might not always be the frontier option. They might not top the leaderboards. But they’re yours. No government order, no policy change, no business dispute can take them away from you. Think of it like owning a generator versus relying on the grid. The grid is more powerful, sure. But when it goes dark, you’re the one still running.

You don’t have to run everything on open-weight models. But having one in your fallback chain means you always have a floor. A baseline that works regardless of what happens to your commercial providers.

This is basic resilience engineering, but I’m amazed how few teams implement it for their LLM calls. When your AI provider starts failing, you need to detect it fast, stop sending traffic, and route to an alternative. Don’t wait for timeouts to cascade through your system.

The pattern is simple: monitor error rates, trip the breaker when they spike, route to your fallback, and periodically check if the primary is back. We do this for every microservice. Your model endpoint deserves the same treatment.

When Anthropic pulled Fable 5 and Mythos 5, you know what kept running? Opus 4.8. A slightly older, slightly less capable model. But it worked.

That’s the pattern. A smaller or older model serving a slightly degraded experience is infinitely better than a broken application serving nothing. Design your system so it can drop down a tier without crashing. Your users would rather get a good-Enough response than an error page. I touched on the non-deterministic nature of LLMs before and how we’re still figuring out how much to trust them. Graceful degradation is part of that answer.

I’ve been talking about Day 2 operations for GenAI workloads for a while now. And the core message hasn’t changed: treat your AI components like any other critical production dependency. Observability, failover, and testing what happens when things break. All of it applies.

Werner Vogels has been saying “everything fails all the time” for years. Your AI provider will have a disruption. It might be an outage. It might be a pricing change that makes your unit economics impossible overnight. It might be a model deprecation with a 30-day notice. Or it might be a government letter on a Friday afternoon.

So ask yourself: does your architecture assume this will happen?

If the answer is no, this week gave you a preview of what’s coming. And next time, it might be your provider.

Have you built multi-provider fallback into your AI stack? Or are you still running on hope-driven architecture? Let me know in the comments below.