Feature engineering

What is feature engineering?

Feature engineering turns raw columns into the inputs a model can actually learn from. The columns a model reads during training and at prediction time are its features, and most raw data does not arrive in that shape. A table of orders with a customer number, a date, and an amount holds the signal you want, but no single row says "this customer is about to leave".

So you build features. From that same orders table you can derive days since the last order, average order value over the last quarter, or the number of support tickets in the past six months. Each one collapses hundreds of raw rows into a single number that means something to a machine learning model. A good feature carries more signal than the column it came from, because you have folded your knowledge of the business into it.

That is why teams spend more time shaping data than tuning algorithms. The model is often a standard choice; the features are where the work and the judgement sit.

Common techniques, with examples

Most of feature engineering comes down to a handful of moves you reach for in almost every tabular project. The names sound technical; the operations are not.

• Windowed aggregations. Count, sum, or average events inside a time window: orders in the last 90 days, revenue this quarter, logins in the past week. One row per customer, one number per window.

• Ratios and derived values. Divide one column by another to expose a trend a raw column hides: this quarter's revenue over last quarter's, returns over orders, margin per order.

• Date parts. From one timestamp you pull the weekday, month, quarter, and whether it was a holiday. Anything with a seasonal pattern leans on these.

• Categorical encoding. A model works on numbers, not words. A sector column with values like retail, construction, and hospitality becomes one yes/no column per value. That is one-hot encoding, OneHotEncoder in scikit-learn.

• Binning. Cut a continuous value such as age or order size into ranges, so the model treats "under 30" and "over 65" as groups rather than raw numbers (KBinsDiscretizer).

• Text and interaction features. Turn free text into counts or embeddings, or multiply two columns together when their combination matters more than either alone (PolynomialFeatures builds these interaction terms).

A windowed aggregation, and the rows it may see

Take a churn model that scores every customer on the first of the month. For a training example dated 1 March 2026, you want the feature orders_last_90d: the number of orders that customer placed in the 90 days before that date.

The window matters more than it looks. That feature may only count orders dated between 1 December 2025 and 28 February 2026, the 90 days that end the day before the cutoff. It must not see a single order dated 1 March or later, because on the morning you would really make this prediction, those rows do not exist yet.

This is the point-in-time correctness rule: a feature may only use data that was available at prediction time. Break it and you have data leakage, where the feature quietly contains the answer. Models built on leaked features look excellent in testing and then fall apart in production, because the information they relied on is not there when it counts.

Training-serving skew: the real risk

The failure that costs the most is subtler than a bad feature. It is the same feature computed two different ways. In a training notebook you might calculate average order value over a calendar quarter, while the production service that scores live customers computes it over a rolling 90 days. Neither is wrong on its own, but the model was trained on one definition and now sees the other. Its accuracy drops in production and nobody can say why.

Google's Rules of Machine Learning names this training-serving skew and gives two blunt defences: re-use the same code between your training and serving pipelines, and log the exact features used at serving time so you can train on them later. The aim is one definition of each feature, not two that drift apart. This is squarely an MLOps concern, sitting next to model serving and experiment tracking in the work around a model.

The feature store as one answer

A feature store is one way to hold that single definition. You define a feature such as 7day_transactions_sum once, and the store serves the same computation to both the training job and the live model, which is how it removes training-serving skew. Azure's managed feature store and Databricks Feature Store both work this way.

It also handles the point-in-time problem for you. A feature store does a point-in-time join, sometimes called time travel, matching each label to the feature values as they stood at that moment, so future data never leaks into the training set. For one model that retrains monthly you do not need any of this: a documented calculation in your data pipeline does the job. Once several models and teams share the same features, a store earns its place.