Open-weight model

What is an open-weight model?

An open-weight model is an AI model whose trained weights can be downloaded. The weights are the learned parameters of the model: the numbers that were adjusted during training and now drive its behaviour.

When you can download the weights, you can run the model on your own hardware, in your own cloud environment, or through a hosting provider you choose. You are not forced to call the original maker's API for every prompt.

Open-weight models sit between fully closed hosted models and fully open-source AI systems. Examples of model families with downloadable weights include Llama, Gemma, Mistral, Qwen, Phi, and many community models, although the licence and allowed use differ per release.

Open weights are not automatically open source

Open weights mean you can get the finished model parameters. That does not mean you get the training data, the full training code, the exact data filtering process, or permission to use the model however you like.

The Open Source Initiative draws that line clearly in its Open Source AI Definition. A genuinely open-source AI system needs more than downloadable weights: it needs enough information to understand, modify, and recreate the system, including the code and meaningful information about the training data.

A simple way to remember it: open weights give you the trained model. Open source should give you enough of the recipe to inspect and rebuild it.

For most business teams, the practical consequence is licence review. Do not rely on marketing labels such as open or community. Read the model card and licence for the exact version you plan to use.

Open-weight model versus API model

Privacy and control. With a hosted API, prompts and outputs travel to the provider's service. With self-hosted open weights, the model can run inside your own environment. That can simplify data-protection conversations, but it also makes you responsible for security.

Cost shape. API models are usually priced per token or per request. Open-weight models move the cost to hardware, cloud GPU time, storage, monitoring, and staff time. Low volume often favours an API. Steady high volume can make self-hosting attractive.

Operations. An API provider handles scaling, uptime, updates, safety filters, and infrastructure. With open weights, your team owns deployment, patching, model serving, logging, capacity, and incident handling.

Capability. The strongest hosted models may outperform a smaller self-hosted model on broad reasoning. A smaller open-weight model can still win on a narrow task after fine-tuning or careful prompting. Test on your own examples.

Licences and model cards

The licence decides what you may do with an open-weight model. Check at least these points:

• Commercial use: is it allowed for your company and your use case?

• Redistribution: may you share the model, a fine-tuned version, or an application that packages it?

• Attribution and naming: must you mention the original model or include certain wording?

• Usage restrictions: are there prohibited domains, safety rules, or acceptable-use policies?

• Scale thresholds: do extra terms apply above a certain company size, revenue, or user count?

The model card is just as important. It should describe intended use, limitations, evaluation results, training-data notes, safety notes, and technical requirements. If the model card is thin, treat the model as higher risk.

Where open-weight models run

Small models can run on a laptop or workstation through tools such as Ollama, LM Studio, llama.cpp, or Microsoft Foundry Local. This is useful for testing, demos, private prototypes, and edge scenarios.

Larger models need GPUs or specialised accelerators. You can use your own server, rent cloud GPUs by the hour, or use a managed inference platform that hosts the open-weight model for you.

Running the model is only part of the stack. You may also need a vector database or search index for RAG, a prompt layer, monitoring, access control, evaluation, and a way to roll back when a new model version behaves differently.

When open weights make sense

• Sensitive data. You want prompts, documents, and outputs to stay inside your own environment.

• Predictable high volume. A fixed serving cost can beat per-token pricing once usage is steady enough.

• Fine-tuning or domain adaptation. You want to adapt the model to your terminology or task and keep the result under your control.

• Offline or edge use. The model needs to run on a machine, device, or network that cannot depend on an external API.

• Vendor independence. You want the option to keep running a model even if a provider changes prices, terms, or availability.

What to watch out for with open-weight models

Self-hosting is not free. Hardware, GPU time, electricity, deployment, monitoring, and support all count.

The licence can change per version. One release may be permissive, another may have extra restrictions. Review every upgrade.

Security becomes your job. Model servers, downloaded files, dependencies, prompt logs, and admin interfaces need the same care as any other production system.

Quality needs measurement. Build a small evaluation set with your own tasks before switching models or fine-tuning.

Open weights do not explain the training data. If copyright, privacy, or bias risk matters, downloadable weights alone do not answer those questions.