ArticlesProjectsWeeklyCredentialsAbout

TMC #0014: Hebb Learning Rule: Associative Memory Network

A pure-Python implementation of Hebbian learning (1949): outer-product weight storage, bipolar associative recall, noise-tolerance testing, capacity limit demonstration, and Oja's normalised rule for comparison.

hebbian-learningassociative-memoryneural-networksoja-rulepythonneuroscience

Hebb's 1949 outer-product rule is the oldest weight-update algorithm in neural network history. This project implements it faithfully, tests its associative recall under noise, and measures the capacity limit (pmax⁡≈0.138np_{\max} \approx 0.138n) empirically.

What It Includes

HebbianNetwork: nn-neuron bipolar network. store(pattern) applies the outer-product update W+=xxT/nW \mathrel{+}= \mathbf{x}\mathbf{x}^T / n; recall(cue, steps) performs synchronous threshold updates until convergence. Diagonal is zeroed (no self-connections).

Noise tolerance demo: Stores 5 patterns in a 100-neuron network, then queries each at corruption levels 5%–40%. Reports recall accuracy at each noise level and shows the cliff where retrieval fails.

Capacity limit experiment: Sweeps the number of stored patterns from 1 to 30 for n=100n=100 neurons, measures average recall error, and finds the empirical pmax⁡p_{\max}: showing the theoretical ≈13–14\approx 13–14 pattern limit for n=100n=100.

Oja's rule comparison: OjaNetwork implements Δwij=η(aiaj−aj2wij)\Delta w_{ij} = \eta(a_i a_j - a_j^2 w_{ij}); weight norms are plotted alongside pure Hebb to show stabilisation vs unbounded growth.

Running It

python hebb_sim.py                   # all demos
python hebb_sim.py --demo recall     # noise tolerance only
python hebb_sim.py --demo capacity   # capacity limit sweep
python hebb_sim.py --demo oja        # Oja normalisation comparison

No external dependencies: pure Python 3.10+.

Source code
# TMC #0014 — Hebb's Learning Rule: Associative Memory Network

Companion code for [The Thinking Machine Chronicles #0014](/articles/tmc-0014-hebb-learning-rule).

Implements Hebbian outer-product learning, noise-tolerance testing, empirical capacity limit demonstration, and Oja's normalised rule.

## Usage

```bash
python hebb_sim.py                  # all demos
python hebb_sim.py --demo recall    # noise tolerance only
python hebb_sim.py --demo capacity  # capacity limit sweep
python hebb_sim.py --demo oja       # Oja normalisation comparison
```

Pure Python 3.10+, no external dependencies.
← All projects