Start Here: The Maths Behind Machine Learning

If you have decided to learn machine learning and immediately hit a wall of unfamiliar notation — vectors, gradients, sigmas — you are in the right place. This is the order we recommend working through the material on this site. It assumes GCSE-level algebra and nothing else.

1. 0

   Orientation

   Get your bearings

   Before any maths, it helps to know what a model actually is, and what the three broad families of machine learning are for.

   • What is a machine learning model?Start here — parameters, features, labels, and how training differs from inference
   • A Beginner’s Guide to Machine Learning Basics — what “learning from data” really means
   • Supervised vs Unsupervised vs Reinforcement Learning — which problem type you are actually facing
   • 7 Essential Supervised Classification Algorithms — the models you will meet first
2. 1

   Foundations · Maths

   Linear algebra: how data is represented

   Every dataset you will ever load is a matrix. Every row is a vector. Almost all of the speed in modern machine learning comes from expressing operations on those objects instead of looping over them.

   1a · Vectors

   • Vectors vs scalars vs matrices — the vocabulary, first
   • Adding vectors and subtracting vectors
   • Multiplying vectors: 5 methods — dot product vs cross product vs elementwise
   • Vector norms — how you measure “size”, and why L1 and L2 disagree
   • Unit vectors — direction without magnitude
   • Orthogonal vectors — what “unrelated” looks like geometrically

   1b · Matrices

   • Matrix multiplication dimensions — the rule behind most NumPy shape errors
   • Transpose and the identity matrix
   • Determinants and the inverse of a matrix
   • Rank of a matrix — how much independent information your features carry
   • Linear independence and span

   1c · Where it pays off

   • Eigenvectors and eigenvalues — the idea behind dimensionality reduction
   • The covariance matrix — how features vary together
   • Singular value decomposition (SVD)
   • Step-by-step PCA with NumPy — all of the above, applied
   • Linear algebra for machine learning — the summary, once the pieces make sense
3. 2

   Foundations · Maths

   Calculus: how models improve

   Training is repeated small adjustments in whichever direction reduces error. Derivatives are how a model works out which direction that is. You need far less calculus than a maths degree demands — mostly derivatives and the chain rule.

   • What is a limit? — the foundation, briefly
   • Properties of limits and continuity
   • Limits and continuity for ML — why a loss function needs to be smooth
   • Derivative of a fraction — the quotient rule in practice
   • The chain rule for machine learningStart here — backpropagation is the chain rule applied repeatedly. The single most important item on this page.
   • Logarithms — you will meet these inside every cross-entropy loss

   Trig derivatives appear less often in machine learning than in a calculus course, but if you need them: trig derivatives, sin, cos, tan, cot, csc. For approximating awkward functions: Taylor series and Maclaurin series.

4. 3

   Foundations · Maths

   Probability and statistics: how you judge results

   Statistics is what stops you fooling yourself. It is the difference between a model that looks good on your screen and one that works on data it has never seen.

   • Statistics for machine learning — start with this overview
   • Bessel’s correction — why sample variance divides by n−1
   • Covariance vs correlation, and reading a negative covariance
   • Interquartile range — a robust way to spot outliers
   • Odds ratios — needed to interpret logistic regression coefficients
5. 4

   Machine learning

   How a model actually learns

   This is where the maths turns into machine learning. A loss function scores how wrong the model is; gradient descent uses the derivative of that score to adjust the weights; activation functions let a network represent something other than a straight line.

   4a · Loss functions — scoring the error

   • Loss functions in ML: 8 types and how to chooseRead first
   • MSE, MAE, RMSE, Huber loss — for regression; Huber when you have outliers
   • Cross entropy, binary cross entropy, hinge loss — for classification

   4b · Gradient descent — using the error

   • Gradient descent, worked through logistic regression
   • The vanishing gradient problem — what goes wrong in deep networks, and the fixes
   • Feature scaling — why unscaled features make gradient descent crawl

   4c · Activation functions — adding non-linearity

   • Activation functions: the complete guide
   • Sigmoid, tanh, ReLU, Leaky ReLU
   • Softmax, and when to use sigmoid instead
6. 5

   Machine learning

   Evaluating a model honestly

   Accuracy is the most misleading number in machine learning. On a dataset where 99% of cases are negative, a model that always predicts “negative” is 99% accurate and completely useless. These pages explain what to look at instead.

   5a · Getting numbers you can trust

   • Training data vs test dataRead first — never judge a model on the data it learned from
   • Overfitting and underfitting — diagnosing which of the two ways your model is failing

   5b · Reading the metrics

   • The confusion matrixRead first — every other metric is derived from this table
   • Precision and recall — the two numbers accuracy hides
   • F1 score — balancing the two
   • ROC curve and AUC — judging a model across every threshold
   • Classification metrics: 5 to know and when to use them
7. 6

   Build it

   Build something

   Start with linear regression — you can work the whole thing out on paper. Then logistic regression, which uses nearly everything above: a dot product, a sigmoid, a cross-entropy loss and gradient descent.

   • Linear regression explainedBuild this first — least squares, worked by hand on four data points
   • Logistic vs linear regression — what changes when the answer is a category
   • A worked logistic regression example
   • Logistic regression in scikit-learn — the same thing in code
   • The decision boundary — seeing what the model learned
   • Assumptions to check — before trusting the output
   • Your first Hugging Face model — for when you want a result today