From a Pyodide Wrapper to an Edge AI Engine

Source: Dev.to

A few months ago, I shipped v1 of a React library that let you run Python ML models directly in the browser. It worked — but the feedback was blunt and fair:

“This is basically just a wrapper around Pyodide.”

That line mattered, because it exposed the real problem.

What v1 got right (and wrong)

v1 proved feasibility

• Python + WASM + React can work client‑side.

Issues in real apps

• Inference froze the UI.
• Workers outlived React components → memory leaks.
• WASM OOMs caused silent hangs.
• Cleanup logic was fragile and scattered.

Running Python wasn’t the hard part; orchestrating failure was.

What changed in v2

• Full rewrite – shifting from a wrapper to an infrastructure layer for Edge ML.
• Supervised worker pools – hung or OOM workers are terminated and replaced automatically.
• Strict React lifecycle binding – component unmount = worker termination. No zombies.
• Zero‑copy data paths – SharedArrayBuffer where possible to avoid serializing large tensors.

The goal wasn’t speed alone — it was predictability under failure.

Where this is going

I’m now prototyping a Neural Bundler — a build‑time compiler that translates Python math logic into WebGPU compute shaders, skipping Pyodide entirely for math‑heavy workloads. Still early, but very promising.

If you’re building client‑side ML, edge inference, or anything involving WASM + React, I’d love your thoughts.

Repository link