Show HN: The Hessian of tall-skinny networks is easy to invert

Published: 13 hours ago (January 15, 2026 at 03:36 PM EST)

1 min read

Source: Hacker News

Overview

It turns out the inverse of the Hessian of a deep net is easy to apply to a vector. Doing this naively takes cubically many operations in the number of layers (so impractical), but it’s possible to do this in time linear in the number of layers (so very practical)!
This is possible because the Hessi…

Back to Blog

Rethinking Learning Dynamics in AI Models: An Early Theory from Experimentation

Observing Representation Instability During Neural Network Training While experimenting with neural network training behaviors, I noticed a recurring pattern t...

Reproducing DeepSeek's MHC: When Residual Connections Explode

Article URL: https://taylorkolasinski.com/notes/mhc-reproduction/ Comments URL: https://news.ycombinator.com/item?id=46588572 Points: 14 Comments: 6...

Teaching a Neural Network the Mandelbrot Set

And why Fourier features change everything The post Teaching a Neural Network the Mandelbrot Set appeared first on Towards Data Science....

Data Analyst Guide: Mastering Neural Networks: When Analysts Should Use Deep Learning

Data Analyst Guide: Mastering Neural Networks – When Analysts Should Use Deep Learning As a data analyst, you're likely familiar with the buzz surrounding neur...

Overview

Related posts

Rethinking Learning Dynamics in AI Models: An Early Theory from Experimentation

Reproducing DeepSeek's MHC: When Residual Connections Explode

Teaching a Neural Network the Mandelbrot Set

Data Analyst Guide: Mastering Neural Networks: When Analysts Should Use Deep Learning