ArticlesProjectsWeeklyCredentialsAbout

Gradient Descent from Scratch in Python

Vanilla SGD, momentum, RMSProp, and Adam implemented from scratch in NumPy — with a shared loss surface visualisation so you can watch each algorithm find the minimum differently.

machine-learningoptimizationdeep-learning

Four optimisers, one loss surface — SGD, momentum, RMSProp, and Adam in plain NumPy with convergence plots.

Source code
# Gradient Descent from Scratch

Code from the article [Gradient Descent: The Algorithm That Trains Every Model](https://rishisharma.in/articles/gradient-descent).

Requires Python 3.8+ and NumPy. Matplotlib is optional (for plots).

```sh
pip install numpy matplotlib
python optimisers.py
```

## Contents

- `optimisers.py` — SGD, momentum, RMSProp, and Adam implemented in NumPy with convergence comparison
- `loss_surface.py` — 2-D Rosenbrock loss surface used as a benchmark
← All projects