Binary cross entropy is the loss you reach for whenever a model must answer a yes-or-no question: spam or not, fraud or not, click or no click. It is the two-class special case of cross-entropy, and it is also known as log loss. It compares a single predicted probability against a 0-or-1 label and, like its multi-class parent, it forgives confident correct answers and severely punishes confident wrong ones.
What is binary cross entropy?
For each example there is a true label that is either 1 (yes) or 0 (no), and the model outputs a single probability p that the answer is yes. BCE rewards the model when p is high and the label is 1, or when p is low and the label is 0, and it penalises the opposite. Because the label is either 1 or 0, only one half of the formula is ever active at a time: when the truth is yes you are charged for how little probability you put on yes, and when the truth is no you are charged for how little you put on no. That symmetry is what makes the loss a fair judge of a two-class model.
The name log loss is used interchangeably, especially in data-science competitions where it is a favourite metric precisely because it rewards honest, calibrated probabilities rather than just correct labels.
The binary cross entropy formula
When the true label y is 1, the second term vanishes and the loss is simply −log(p). When y is 0, the first term vanishes and the loss is −log(1−p). So the loss is really the same negative-log-probability penalty as its parent, written to cover both sides of a yes/no decision in one expression.
Try it: the log-loss calculator
Set the probability the model gave the correct answer and see the penalty. The shape is identical to the multi-class case — gentle near 1, explosive near 0.
Binary cross entropy and the sigmoid
A worked example
Two spam filters both correctly flag a spam email (label = 1). Filter A is 90% sure; filter B is only 55% sure. Their penalties:
Both are “correct,” yet the loss prefers the confident filter A, and it would savage either one for being confidently wrong. This is why it produces classifiers whose probabilities you can actually trust for thresholding and ranking.
Multi-label: many yes/no questions at once
BCE also handles multi-label problems, where several independent tags can each be true — a photo that is both “beach” and “sunset.” You put one sigmoid on each output and apply the loss to each independently, then average. This is exactly the case where you would not use softmax, because the labels do not compete; see sigmoid vs softmax.
Common mistakes to avoid
Binary cross entropy in Python
import numpy as np
def bce(y, p):
p = np.clip(p, 1e-12, 1 - 1e-12)
return -(y*np.log(p) + (1-y)*np.log(1-p)).mean()
y = np.array([1, 0, 1]); p = np.array([0.9, 0.2, 0.7])
print(bce(y, p)) # 0.23Use nn.BCEWithLogitsLoss() in PyTorch (safe, takes logits) or binary_crossentropy in Keras. The Wikipedia cross-entropy article covers the theory.
Putting it into practice
On imbalanced yes/no problems — fraud, disease, rare clicks — the loss is often paired with class weights or focal-loss variants so the rare positive class is not drowned out by the majority. It is also the standard metric on probability-scoring leaderboards, where a model that hedges sensibly beats one that makes over-confident guesses. When you evaluate, pair the loss with a threshold-based metric such as precision and recall, because a low log loss tells you the probabilities are well calibrated but not where you should set the yes/no cut-off. Finally, if the loss refuses to fall, check that your labels really are 0 and 1 and that your final layer is a single sigmoid; a surprising number of stubborn the loss problems come down to a mismatched output shape rather than anything subtle about the loss itself.
Handling class imbalance
Real yes/no problems are often lopsided: fraud, disease and ad-clicks all have far more negatives than positives, and a naive model can score a deceptively low loss by simply predicting “no” almost always. The standard remedy is to weight the two classes so that mistakes on the rare positive class cost more, which most frameworks expose as a single parameter. A more aggressive option is focal loss, a modified objective that automatically down-weights the easy, already-correct examples so training focuses on the hard, informative ones; it was designed for extreme imbalance in object detection and has since spread widely. Whichever route you take, the golden rule is to judge the model on the minority class you actually care about, using recall or precision on that class rather than the overall average, because the averaged number can look healthy while the model ignores exactly the cases that matter.
Turning scores into decisions
A trained model outputs a probability, but a product needs a decision, and the gap between the two is a threshold you must choose deliberately. The default of 0.5 is rarely optimal on imbalanced or asymmetric problems; if a missed positive is far costlier than a false alarm, you lower the threshold to catch more positives at the price of more false alarms, and vice versa. The right cut-off comes from the real-world cost of each error, and tools like the precision-recall curve let you see the trade-off across every possible threshold at once. It is also worth checking calibration: a well-trained probability of 0.8 should be correct about 80% of the time, and if it is not, a quick calibration step can make the probabilities trustworthy enough to threshold with confidence. Keeping the scoring objective and the decision threshold as two separate choices is one of the most useful habits in applied classification.
Why it beats squared error here
A natural question is why classification does not just reuse the squared penalty that works so well for regression. The answer is that squared error, placed on top of a probability output, produces a nearly flat gradient when the model is very wrong — exactly when you most need a strong corrective push — so training crawls. The log-based penalty does the opposite: its gradient grows as the prediction gets more confidently wrong, so the model is shoved hardest precisely where it is failing worst. Squared error also assumes symmetric, bell-shaped noise, an assumption that simply does not hold for a variable that must live between zero and one. Together these reasons explain why the probability-scoring objective, not squared error, became the standard for yes/no models, and why swapping it in almost always trains faster and calibrates better on the same data. In short, the log-based penalty is not a stylistic preference but a structural fit for probability outputs, which is why virtually every yes/no neural network in production is trained this way rather than with a squared error it could technically compute. If you ever inherit a binary model that trains sluggishly and calibrates poorly, checking whether someone quietly used a squared penalty on a sigmoid output is a five-second fix that surprisingly often solves the problem outright. It is a small reminder that, in machine learning, matching the objective to the shape of the output is often more important than the choice of model architecture sitting above it.