Local AI is a pipe dream

Source: Dev.to

Overview

Software runs in a partially observable and changing environment.
We discovered that operating systems are a necessary infrastructure to decouple resource allocation from the application itself. That means for application developers, there is a platform police to run past. Users’ affinity to spend time updating software is never as much as developers want. Building desktop software means fewer and fewer deployments, leading to slower bug discovery, security, and performance issues, slower iterations from experience. Backward compatibility becomes paramount and hard to achieve in a runtime that developers cannot see. All of this results in higher development costs.

The application and OS need an intermediate platform

A good and stable web browser on a common standard solved platform integrations, security, and packaging. Application developers do not have to worry about OS version compatibility. Process‑level optimization and memory were completely abstracted out for the first time, and events became first‑class objects without adding additional complexity. Even today, it’s significantly harder to develop for macOS compared to other apps on the Google Chrome browser.

Decentralized access matters more than decentralized execution

There is a great operational advantage in simply running a web server and connecting to a remote interface. It’s harder to run workloads and computations frequently, going beyond the personal computer’s capabilities. Sometimes, reliable technology requires centralization, such as in instant messaging. Oddly, because of the policies needed to support security in decentralized email, partial centralization is required. It is harder to ask people to pay for software before it has proven its worth. That led to a pricing model where cheaper services became increasingly available on the internet. Historically, this is a time when access matters more than privacy.

People always want a better toy

We now have more compute on watches than my personal computer from high school and the computer on the Apollo mission. People will never run out of work. The tree of imaginative and physical abstractions of nature always grows. There’s an unbelievable compositional ability among the objects in nature and the abstract, on which the market trades forever—unless something drastic happens. As long as consumers upgrade hardware slower than companies can pool significantly larger compute and offer a better toy, building locally is not an obvious choice.