Transfer learning

What is transfer learning?

Transfer learning is reusing a model trained on one task as the starting point for a related task. Instead of training from scratch, you take a model that already learned useful patterns from a large dataset and adapt it to your own, usually smaller, problem. It saves data, time, and compute.

Think of someone who already speaks Spanish picking up Italian. They learn far faster than a beginner starting from nothing, because a lot of the groundwork carries over. Transfer learning does the same thing for models: the general knowledge from the first task gives the second task a head start.

It works because low-level skills are shared across tasks. A vision model trained on millions of photos has already learned edges, textures, and shapes. Those are useful whether you are now spotting defective parts on a production line or sorting product photos in a catalogue. You keep that learned base and teach only the new, specific part.

How transfer learning works

There are two main ways to reuse a pretrained model, and they differ in how much of it you change.

Feature extraction. You freeze the pretrained model and use it as is, letting it turn your inputs into useful features, then train only a small new layer on top for your specific task. It is fast and cheap, and it works well when your dataset is small or very close to what the original model saw.

Fine-tuning. You unfreeze some or all of the pretrained layers and keep training on your own data, so the weights shift to suit your task. It reaches higher accuracy but needs more data and more compute. Fine-tuning is one specific technique within the broader idea of transfer learning, not a separate thing.

Where you see transfer learning

Transfer learning is most common in deep learning, because deep networks learn features in layers, and the lower layers tend to be reusable. It is the engine behind foundation models and large language models: a base model is pretrained on broad data once, then countless teams adapt it to their own tasks rather than each training a giant model from scratch. The same pattern runs through image classification with a pretrained backbone, speech recognition, and many other areas where good labelled data is scarce.

A related but different idea is model distillation, where a large model teaches a smaller one to mimic it. Distillation moves behaviour into a smaller model to cut running costs; transfer learning moves knowledge into a new task. They are sometimes combined.

Transfer learning versus training from scratch

Training from scratch gives you full control and a model shaped entirely by your data, but it demands a large labelled dataset and serious compute, which many teams do not have. Transfer learning gets you a working model with far less data and far less time, which is why it is the default starting point for most image and language work today.

The trade is that you inherit the source model. Its assumptions, its blind spots, and any bias in its original training data come along, so an adapted model still needs testing on your real cases.

What to watch out for with transfer learning

The tasks have to be related. Transfer works when the source and target share structure. Reuse a model on something too distant and the borrowed knowledge can actively hurt, a problem known as negative transfer.

You inherit the source's flaws. Gaps and bias in the original training data carry into your model. Adapting it does not scrub out what it learned before.

Fine-tuning too hard erases skill. Push the training too aggressively and the model forgets the general abilities that made it worth reusing. Test on a broader set than your own task before trusting it.

Check the licence. Pretrained weights come with terms. Confirm you are allowed to use a given model for your purpose, especially commercially, before you build on it.