Author name: Jamal Hussein

The root mean squared error (RMSE) is the most widely reported single number for regression quality, and for one simple […]

The mean absolute error (MAE) measures how wrong a regression model is by averaging the size of its mistakes, ignoring

The mean squared error (MSE) is the single most common way to measure how wrong a regression model is. It

Activation functions are the small non-linear functions applied at every neuron that give neural networks their power to learn curves,

The sigmoid vs softmax choice decides your network’s output layer, and getting it wrong quietly breaks a classifier. The short

The vanishing gradient problem is the reason deep neural networks were nearly impossible to train for years. As error gradients

The ELU activation function — the Exponential Linear Unit — behaves like ReLU for positive inputs but replaces the flat

The Swish activation function is a smooth, self-gated activation defined simply as the input times its own sigmoid. Found by

The GELU activation function — the Gaussian Error Linear Unit — is the activation behind modern transformers such as BERT

Leaky ReLU is a simple, popular fix for the biggest weakness of ordinary ReLU: dead neurons. Instead of flattening every