Every other post on this site covers one piece. This one puts them in order, so you can work through a real dataset start to finish without wondering what comes next.
Step 1 — Frame the question first
Before any code: what are you predicting, and what does a useful answer look like? “Predict house prices” is a regression problem. “Will this customer churn” is classification. The answer decides your model and your metrics.
Also decide what “good enough” means now, not after you see the results. It is remarkably easy to move the goalposts to wherever your model happened to land.
Step 2 — Load and explore
Open the data in pandas and run the five commands: shape, head(), info(), describe(), isna().sum().
Step 3 — Split, before anything else
This is the step beginners get out of order, and it is the one that matters most.
⚠️ Split now, not after cleaning
Split first. Everything that learns from data gets fitted on the training portion only.
Step 4 — Preprocess, fitted on training data
In order: fill the gaps, encode the categories, deal with outliers, then scale. This is also where feature engineering earns its keep — usually more than any model choice you make later.
Wrap it all in a scikit-learn Pipeline. It is not tidiness for its own sake: a pipeline makes it structurally impossible to leak, because every step is refitted inside each cross-validation fold automatically.
Step 5 — Train a baseline first
🧪 Start deliberately stupid
That number is your floor. If a neural network beats it by two points, the neural network is not working — and without the baseline you would have called 88% a success.
Then fit something simple and real: linear or logistic regression, or a decision tree. Only then reach for anything complicated.
Step 6 — Evaluate honestly
Use cross-validation on your training data to compare options, and report the spread as well as the mean.
Pick metrics that match the problem: RMSE or MAE for regression; the confusion matrix and precision and recall for classification. Accuracy alone is rarely enough and is actively misleading on imbalanced data.
Compare training against validation scores to see whether you are overfitting or underfitting, and fix whichever you have before doing anything else.
Step 7 — Test once, then ship
When everything is settled, evaluate on the test set. Once. That number is what you report.
If it is much worse than your cross-validation scores, you have overfit to your own decisions — and the honest response is to say so, not to go back and tune further against the test set.
Before it goes anywhere near real users, check performance per group, not just overall. Then see deploying your first model.
A realistic sense of where the time goes
| Stage | Share of effort | What beginners expect |
|---|---|---|
| Understanding the problem | ~15% | Skipped entirely |
| Cleaning and features | ~60% | A quick preliminary |
| Modelling | ~15% | The whole job |
| Evaluating and writing up | ~10% | An afterthought |
Good first datasets
Pick something small and well-understood so you are learning the process, not fighting the data. The Titanic dataset covers missing values and categorical encoding. California Housing is a clean regression problem that ships with scikit-learn. Palmer Penguins is a friendlier replacement for the Iris dataset.
Whichever you choose, write up what you did and what you found. Explaining your reasoning is what turns a tutorial you followed into something you actually understand.
🔑 Key Takeaways
- Split before you preprocess. Everything that learns from data is fitted on training data only.
- Always build a trivial baseline first, so you know what “good” actually means.
- Use a pipeline — it makes leakage structurally impossible.
- Touch the test set once, at the very end.
- Expect to spend most of your time on data, not on models.
Further reading
scikit-learn’s introductory tutorial follows this same order in code, and its pipeline documentation shows how to bundle preprocessing and model into one object.
Where to go next
- Random forests — the next post, and the best default for table data
- Cross-validation — how to compare options honestly
- Deploying your first model — what happens after step 7
- Start Here — the full learning path