AutoML (automated machine learning)

What is AutoML?

AutoML is software that does the repetitive part of building a machine learning model for you. You hand it a table with one column marked as the thing you want to predict. It then preprocesses the other columns, tries a list of algorithms, searches through their settings, checks each candidate on data it has not trained on, and hands you a ranked list of models with a score next to each one. The person running it does not write the training code.

The idea is old by AI standards. The Freiburg research group behind auto-sklearn published the approach in 2015, and H2O shipped its AutoML in 2017. What changed is that every cloud platform now has a version built in, so a Power BI analyst or a controller who knows some Python can run one from a notebook.

A useful way to picture it: a hand-built model is a tailor measuring you. AutoML is a rack of fifty suits in every cut, with a mirror. It gets you dressed faster. It does not tell you which occasion you are dressing for.

Where you find it

Microsoft Fabric exposes AutoML in notebooks through the flaml.AutoML class, with every trial logged to MLflow so you can see how the winner was chosen. Azure Machine Learning has automated ML with a no-code studio, and adds voting and stacking ensembles as the last iterations of a run. Google Vertex AI has AutoML for tabular classification, regression and forecasting. AWS SageMaker Autopilot does the same on AWS, and since November 2023 its no-code interface lives in SageMaker Canvas. Outside the clouds, H2O AutoML and auto-sklearn are open source and run on your own machine.

What AutoML automates and what it leaves to you

Every vendor page describes the automated part in roughly the same words. Missing values get filled, text and categories get encoded into numbers, numbers get scaled. A list of algorithms is trained: logistic regression, random forests, gradient-boosted trees, sometimes a small neural network. For each one, the hyperparameters are searched within a time budget you set. Candidates are compared on held-out data, through cross-validation or a validation split, and ranked on the metric you chose. Most tools also produce a feature importance chart and, on Azure and SageMaker, a model explanation report.

The list of what stays with you is shorter but heavier.

• The question. AutoML does not know whether you want to predict who leaves, who pays late, or who is worth a phone call. Those are three different models with three different tables behind them.

• The target column. The tool predicts whatever column you point at. If that column is badly defined, the model is a precise answer to the wrong question.

• The data. Pulling the table together from the ERP, the CRM and the helpdesk, and deciding which cut-off date each row is frozen at, is most of the work and none of it is automated.

• The cost of being wrong. A false positive means a sales call to a happy customer. A false negative means a customer who leaves without anyone noticing. Which one hurts more decides which model you pick, and the tool cannot know.

• Deployment and monitoring. The leaderboard is a list of files. Scoring new customers every Monday, and noticing when the scores stop being right, is a separate job. The MLOps and model drift entries cover that side.

What usually wins on tabular business data

For the kind of data an SME has, rows of customers or invoices with a few dozen numeric and categorical columns, the winner is boringly predictable. Gradient-boosted decision trees, in the LightGBM, XGBoost or CatBoost flavour, sit at or near the top of the leaderboard most of the time, with random forests close behind.

There is data behind that. A 2022 study by researchers at Inria compared tree-based models against deep learning across 45 tabular datasets and found that tree-based models remain the best choice on medium-sized data of around ten thousand rows, mainly because they cope better with columns that carry no signal and with the threshold-like patterns business data tends to have. The vendors reached the same conclusion in practice: FLAML, the engine behind Fabric, starts its default estimator list with LightGBM and XGBoost, and H2O AutoML puts a full set of pre-configured GBM and XGBoost models, plus grid searches over them, in every run.

So you rarely need a wide search. A few tree models with a decent hyperparameter sweep and one linear model as a readable baseline covers most business tables.

Worked example: churn at a subscription business

A company sells a monthly software subscription to 8,000 business customers and loses roughly ten percent of them a year. The owner wants a list of customers to call before they cancel.

The target. Someone has to decide what churn means. The team settles on: the customer cancels within the 90 days after a snapshot date. They build one row per customer per quarter over the last two years, each row frozen at the start of that quarter, which gives about 60,000 rows with around 1,500 positives. That is a positive rate of 2.5 percent, so the classes are heavily imbalanced.

The features. Twenty columns, all known on the snapshot date. From the ERP: contract age in months, plan tier, monthly amount, number of price changes, invoices paid late in the last six months, days since the last upgrade, discount percentage, payment method, number of products, sector and company size. From the helpdesk: tickets in the last 90 days, tickets in the last year, average resolution time, reopened tickets, category of the last ticket, days since the last ticket, and the most recent satisfaction score.

The first run. The first leaderboard shows a model at 0.98 AUC. That is not good news. A column called contract_end_date had slipped into the table and it is only filled in after a customer gives notice. AutoML found the leak in seconds and built the whole model on it. The column goes, the run is repeated.

The second run. After an hour of search the leaderboard reads: a LightGBM model at 0.82 AUC, an XGBoost model at 0.81, a random forest at 0.80, a gradient-boosted model with limited tree depth at 0.81, and a logistic regression at 0.78. The tool's imbalance handling kicked in on its own: Azure automated ML, for example, checks for class weighting whenever the smaller class is at or below 20 percent of the larger one.

The choice. The winner is the LightGBM model. The one that gets deployed may still be the depth-limited tree model at 0.81, or even the logistic regression. The sales team will call the top 200 names each quarter, and the difference between 0.82 and 0.81 is a handful of names in that list. Against that, the simpler model is easier to explain when a rep asks why a customer is on the list, needs fewer columns kept fresh every week, and is less likely to have learned something odd about one sector. The leaderboard ranks on a metric. The decision is made on a business number: how many churners the top 200 calls actually catch, and what a call costs.

AutoML versus a hand-built model

Compare the two on one dimension: what the person still has to decide.

With a hand-built model, a data scientist decides everything. Which algorithm, which encoding for each column, which settings to try, how to split the data, which metric to optimise, and when to stop. Each decision is a place to make a mistake and a place to add domain knowledge. The output is a model plus a person who understands why it works.

With AutoML, the algorithm, the encoding, the settings, the split mechanics and the stopping rule are taken off the person's plate. What is left is the target definition, the table, the metric, the leakage check and the deployment decision. Those five were also the five hardest items on the data scientist's list. AutoML removes the part that needed coding skill and keeps the part that needs business knowledge.

The honest comparison is therefore AutoML plus a domain expert who asks the right questions, against a data scientist who has to ask the same questions and write the code as well.

AutoML versus a language model

AutoML lives in the world of tabular prediction: a number or a category per row, learned from thousands of past rows. A large language model lives in the world of text. Asking an LLM for churn scores from 20 numeric columns is the wrong tool for the job: you get plausible-looking numbers with no training on your history, at a far higher cost per row than a tree model that scores thousands of rows in a second. The reverse also holds. Fine-tuning a language model on your support tickets is a real technique with its own entry, and it is not AutoML, even though some vendors now put both behind the same button.

Where the two meet is in the features. An LLM can turn the free text of a helpdesk ticket into a category or a sentiment score. That score then becomes one of the 20 columns the AutoML run works with.

What to watch out for with AutoML

A great score on a leaked column. AutoML will exploit any column that quietly contains the answer, and it will do so faster than a person would. Any leaderboard that looks too good is a leakage check first, a celebration second. The data leakage entry lists the usual ways it happens.

A great score on a useless target. A model that predicts which customers cancelled last quarter is trivial once the quarter is over. The target has to be something you can act on before it happens, with features frozen before it happens.

Imbalance hidden behind accuracy. With 2.5 percent positives, a model that says nobody churns is 97.5 percent accurate. Choose a metric that survives imbalance, such as AUC or precision at the number of calls you can actually make, before the run starts.

A model nobody can explain to the sales team. A stacked ensemble at the top of the leaderboard is hard to defend in a meeting. If the reps do not trust the list, they will not call it. Feature importance and explanation reports help, and so does picking the simpler runner-up.

Chasing the leaderboard. Another hour of search may lift AUC from 0.82 to 0.83. It will not change the top 200 much, and the same hour spent on a better feature from the helpdesk usually does more.

The practical rule for an SME is short. Use AutoML to get the first model on the table fast. Put a person on the target definition, the leakage check and the deployment, because those are the three places where the tool cannot help and where the model is won or lost.