Author name: Jamal Hussein

Huber loss is the clever compromise between the two great regression losses. It behaves like the smooth, fast-training mean squared

The root mean squared error (RMSE) is the most widely reported single number for regression quality, and for one simple

The mean absolute error (MAE) measures how wrong a regression model is by averaging the size of its mistakes, ignoring

The mean squared error (MSE) is the single most common way to measure how wrong a regression model is. It

Activation functions are the small non-linear functions applied at every neuron that give neural networks their power to learn curves,

The sigmoid vs softmax choice decides your network’s output layer, and getting it wrong quietly breaks a classifier. The short

The vanishing gradient problem is the reason deep neural networks were nearly impossible to train for years. As error gradients

The ELU activation function — the Exponential Linear Unit — behaves like ReLU for positive inputs but replaces the flat

The Swish activation function is a smooth, self-gated activation defined simply as the input times its own sigmoid. Found by

The GELU activation function — the Gaussian Error Linear Unit — is the activation behind modern transformers such as BERT