Encoding Categorical Variables: 4 Proven Methods

⚡ TL;DR: Models need numbers, so text categories must be converted. Use one-hot encoding when categories have no natural order, ordinal encoding when they genuinely do, and target encoding only with care — it leaks easily.

Your dataset has a “city” column containing Leeds, York and Hull. Nearly every algorithm can only do arithmetic, so those words have to become numbers. Encoding categorical variables is choosing which numbers — and the wrong choice invents relationships that do not exist.

The mistake: label encoding unordered categories

Encoding categorical variables compared: a city column as label encoding versus one-hot encoding columns — Label encoding is compact but invents an ordering. One-hot uses more columns and claims nothing false.

Map Leeds → 0, Hull → 1, York → 2 and you have told the model that Hull sits between Leeds and York, and that York is twice Hull. None of that is true. A linear model will take it entirely literally and fit a coefficient as though cities lie on a number line.

✅ The question to ask: do these categories have a genuine order? Small < Medium < Large does. Leeds, York and Hull do not. If there is a real order, encode it as numbers deliberately — that is ordinal encoding, and it is correct. If there is not, use one-hot.

One-hot encoding

Give each category its own column containing 1 or 0. Three cities become three columns, and each row has a single 1. No ordering is implied because no category is numerically larger than another.

In pandas that is pd.get_dummies(df, columns=["city"]); in scikit-learn it is OneHotEncoder, which is the one to use inside a pipeline because it remembers the categories it saw during training.

⚠️ Two things one-hot encoding will do to you

Column explosion. A postcode column with 3,000 values becomes 3,000 columns, most of them almost entirely zeros. Your dataset becomes wide, sparse, and slow — and distance-based models stop working well.

Unseen categories. If a city appears in your test set that was not in training, the encoder has no column for it. Set handle_unknown="ignore" so it produces all zeros rather than crashing in production.

Choosing an encoding

Method	Columns created	Use when
One-hot	One per category	No natural order, fewer than ~15 categories. The default.
Ordinal	One	A genuine order exists and you specify it yourself
Target	One	Many categories and you need compactness — but see the warning below
Frequency	One	How common a category is genuinely carries signal
Leave as-is	—	Modern gradient boosting libraries handle categories natively

Target encoding, and why it is dangerous

Target encoding replaces each category with the average outcome for that category — replace “Leeds” with the mean house price in Leeds. One column, and it carries real predictive signal.

🧪 Why it leaks

You are encoding a feature using the answer. If a category appears exactly once, its “average outcome” is that row’s label — you have copied the target into the features, and the model will look extraordinary in training and fail completely on new data.

The fixes are smoothing (blend the category average towards the overall average, so rare categories barely move) and out-of-fold encoding (compute each row’s encoding using only the other folds). Both are fiddly to get right, which is why one-hot remains the safer default until you need the compactness.

What about tree models?

Decision trees split on thresholds, so label encoding hurts them far less than it hurts linear models — a tree can carve out “code is between 0.5 and 1.5” to isolate a single category. It is inefficient but not wrong.

LightGBM and CatBoost accept categorical columns directly and handle them better than any manual encoding. If you are using those, declare the column as categorical and leave it alone.

Practical rules

Order first. If the categories have a real order, ordinal-encode and specify the order explicitly. Never let the library guess it alphabetically.
Otherwise one-hot, up to about 15 categories.
Group the long tail. With 200 categories where 190 are rare, keep the top 10 and map the rest to “Other”. Simple, robust, and usually loses nothing.
Fit on training data only, exactly as with imputation and scaling. Use a pipeline.
Do not scale one-hot columns. They are already 0 and 1; standardising them produces odd values for no benefit.

🔑 Key Takeaways

Encoding categorical variables badly invents orderings that do not exist.
One-hot for unordered categories; ordinal only when a real order exists and you set it.
One-hot explodes on high-cardinality columns — group the long tail into “Other”.
Target encoding is compact and leaks easily; needs smoothing or out-of-fold computation.
Fit encoders on training data only, and set handle_unknown="ignore".

Where to go next

Detecting and handling outliers — the next post in this sequence
Handling missing data — the previous step in the pipeline
Decision trees — the models least bothered by encoding choices
Start Here — the full learning path