Custom Rust ML

From Web Dev to ML Researcher — a self-guided path through continual learning

Every chapter builds one learning point and produces a running Rust artifact on your machine. No Python, no PyTorch, no black boxes — just Rust and your 2015 iMac.

Pure std at first, then ndarray, then whatever you need.

Phase 0: Foundational Reorientation

What is learning? Build it from scratch with nothing but std.

Chapter 0

What "Learning" Means in This Context

Function approximation by turning knobs. Fit a line to 50 noisy points using only addition and multiplication.

Chapter 1

From Line to Curve — The Leap to Nonlinearity

A single nonlinear function, composed many times, can approximate arbitrarily complex shapes. Fit a sine wave.

Chapter 2

The Computational Graph — Tracking What Affects What

Backpropagation is bookkeeping, not calculus. Build a minimal autograd engine from scratch.

Chapter 3

Tensors — Organizing Numbers for Parallel Computation

Move to the ndarray crate. Reimplement autograd using Array2<f64> and train on 1000 points simultaneously.

Phase 1: Language Models as Mechanisms

Character-level LMs, attention, transformers, tokenization, pre-training.

🔮 Coming in later chapters

Chapters 4–8: next-character prediction, multi-head attention, decoder-only transformer, BPE tokenization, pre-training at small scale.

Phase 2: Adaptation Without Retraining

Fine-tuning, LoRA, prompt tuning, catastrophic forgetting.

Phase 3: Self-Improvement & Verification

STaR, reward models, DPO, reasoning traces, iterative self-improvement loops.

Phase 4: Continual Learning Systems

Experience replay, parameter isolation, model merging, replication of the CL+SI experiment.

Phase 5: The Frontier

Test-time compute, KV cache, Mixture-of-Experts, production architecture, alignment homeostasis.

25+ chapters · Pure Rust, zero Python

Built for a retired web dev with unlimited time and a 2015 iMac