The Machine Learning “Advent Calendar” Bonus 2: Gradient Descent Variants in Excel

Source: Towards Data Science

Gradient Descent Variants

Gradient Descent, Momentum, RMSProp, and Adam all aim for the same minimum. They do not change the destination, only the path. Each method adds a mechanism that fixes a limitation of the previous one, making the movement faster, more stable, or more adaptive. The goal stays the same. The update beco…