Tanh Activation Function: 4 Facts, Formula & Code

The tanh activation function — the hyperbolic tangent — squashes inputs into the range −1 to 1. It is a zero-centred cousin of the sigmoid, which historically made it the preferred choice for hidden layers and keeps it common inside recurrent networks today. This guide covers the tanh activation function’s formula, why zero-centring matters, its derivative, and Python code.

📌

In one line. The tanh activation function maps inputs to (−1,1) and is zero-centred, giving stronger, more balanced gradients than sigmoid.

Try it: interactive tanh plot

The blue curve is the tanh activation function; the orange dashed curve is its derivative, which peaks at 1.0 — four times steeper than sigmoid.

f(x)derivative f'(x)

The tanh activation function formula

tanh(x) = (e^x − e^−x) / (e^x + e^−x)

The tanh activation function — the hyperbolic tangent — maps every input to the range (−1, 1). It is really just a rescaled sigmoid: tanh(x) = 2σ(2x) − 1. The crucial difference is that it is zero-centred.

Why zero-centred output matters

💡

Faster convergence. Because the tanh activation function outputs both negative and positive values around zero, the gradients that flow into the next layer are balanced rather than all-positive (as with sigmoid). That tends to make optimisation converge faster, which is why tanh was long preferred over sigmoid for hidden layers.

The derivative and its limits

tanh'(x) = 1 − tanh²(x)

⚠️

Still saturates. Like sigmoid, the tanh activation function flattens in both tails, so very deep networks built from it still hit the vanishing gradient problem. That is why ReLU is now the default for deep hidden layers, while tanh remains common in recurrent networks (LSTM/GRU gates).

Tanh activation function in Python

import numpy as np

def tanh(x):
    return np.tanh(x)

def tanh_derivative(x):
    return 1 - np.tanh(x)**2

print(tanh(np.array([-2.0, 0.0, 2.0])))
# [-0.964  0.     0.964]

See the Wikipedia hyperbolic functions article for background.

Practical tips

Scaling inputs to roughly zero mean and unit variance keeps a hyperbolic-tangent unit in its steep, responsive middle region and away from the flat tails where learning stalls. Pairing it with careful weight initialisation — Xavier/Glorot was designed with exactly these symmetric, saturating curves in mind — further protects the gradient early in training. In recurrent architectures it still shines because its bounded output keeps hidden states from drifting or exploding across long sequences, which is precisely why LSTM and GRU cells rely on it internally rather than on unbounded alternatives. If you find a deep feed-forward stack training slowly, that is usually the signal to switch the hidden layers to a rectified unit while keeping the bounded curve only where its squashing behaviour is genuinely needed, such as gates or bounded outputs. One more habit worth adopting is to watch the fraction of units sitting in the flat tails during the first few epochs; if most of them saturate immediately, your inputs or initial weights are almost certainly scaled too large, and a quick normalisation step will fix the training slowdown far more reliably than changing the activation itself.

Related activation functions

Activation Functions — the complete guide (hub).
Sigmoid Function — the non-centred sibling.
ReLU — the modern default.
Vanishing Gradient Problem — what tanh still suffers.

Frequently asked questions

What is the range of the tanh activation function?

The tanh activation function outputs values between -1 and 1, and it is zero-centred, unlike sigmoid which outputs 0 to 1.

Is tanh better than sigmoid?

For hidden layers, usually yes, because tanh is zero-centred and has a steeper gradient (peak 1.0 vs 0.25), so it converges faster. Both still saturate, so ReLU is preferred for deep nets.

Where is the tanh activation function still used?

Inside the gates of LSTM and GRU recurrent units, where its bounded, zero-centred output is useful for controlling cell state.