Explore our new Cybersecurity Terms Glossary, explained by our experts.
Cybersecurity Terms Glossary
Open-weight models can be downloaded, inspected, and run privately, which solves several security problems and creates others—starting with safety training that anyone can remove.
What Are the Risks of Open-Weight Models?
An open-weight model is one whose parameters are published for download, so anyone can run it on their own hardware, fine-tune it, and inspect it. Llama, Mistral, Qwen, and DeepSeek releases have made capable models freely available, and for many organisations that is exactly what they need: no data leaves the network, no per-token bill, no dependence on a provider’s roadmap. The security conversation is more interesting than the usual open-versus-closed framing suggests, because open weights genuinely reduce some risks while moving others onto you.
Self-hosting resolves the hardest question in enterprise AI data boundaries: prompts and documents never leave your infrastructure, so there is no processor agreement to negotiate, no international transfer analysis, and no retention policy to verify. You control the version, so a provider cannot change model behaviour underneath a system you have tested. You can inspect, evaluate, and fine-tune without asking permission. For regulated workloads in health, legal, and defence, these are decisive advantages.
The DeepSeek release in January 2025 illustrated how quickly adoption outpaces governance. Within days, researchers at Wiz reported a publicly accessible database belonging to the provider exposing chat history and keys, and Italy’s data protection authority ordered the service blocked over transparency and transfer concerns. Neither issue was caused by the weights being open—the weights themselves were run privately by many organisations without incident—but both showed how an open release drives rapid, unreviewed adoption of the associated hosted service. The lesson for a security team is to separate two decisions: whether to run the weights internally, and whether to use anybody’s hosted version of them.
Mirror models internally, pin versions, and verify hashes against what you reviewed. Load untrusted checkpoints first in an isolated environment with no credentials. Put the inference endpoint behind authentication and network controls—never expose it directly—and apply rate limits and budgets as you would any API. Build your own guardrails, because the model will not refuse reliably and may not refuse at all after fine-tuning. Evaluate behaviour after every fine-tune or quantisation, since both change safety characteristics. Document what you deployed in a model card of your own, recording the base model, the changes, and the evaluations you ran. Finally, test the deployment adversarially; self-hosting removes a provider’s safety layer, and nothing replaces it unless you build it.
Self-hosting simplifies UK GDPR compliance considerably, but it does not remove it: training or fine-tuning on personal data still needs a lawful basis, and a model that memorises records raises the questions described in our note on membership inference. Under the EU AI Act, free and open-source models receive partial exemptions from general-purpose model obligations, though not where a model presents systemic risk, and not for whoever places a high-risk system on the market using it. Fine-tuning a model and deploying it in a regulated use case can make you the provider in law, with all the duties that follow. At Privia Security, we assess self-hosted model deployments, inference infrastructure, and guardrails through our AI Penetration Test Services.
Less constrained, rather than less safe in every sense. Safety behaviour can be fine-tuned away cheaply, and there is no provider pushing fixes for new attack techniques, so the controls have to be yours. Against that, self-hosting removes the data boundary problem entirely and gives you version stability that hosted services cannot. The right answer depends on which risk matters more to you.
Usually not. Open weights means the parameters are downloadable; open source implies a licence meeting recognised criteria, with training code and data available. Several prominent releases restrict acceptable use or impose thresholds on large deployers, and few publish full training data. Read the licence before building a product on one.
GPU capacity sized for your concurrency, an inference server such as vLLM, model storage and version control, monitoring, and someone to patch all of it. Add guardrails, authentication, rate limiting, and logging, which hosted services provide by default. Budget for the operational burden as well as the hardware, because that is where most self-hosting projects stall.
Not conclusively. A trigger-based backdoor leaves aggregate performance unchanged, so evaluation will not reveal it. Mitigate through provenance instead: use well-known publishers, verify artefact hashes, prefer formats that cannot execute code on load, and include targeted probing in your testing. Treat an unknown publisher’s weights as untrusted code, because that is what they are.
You May Be Interested In These
We use a few cookies that are needed to make this website work, and they are always on. We would also like to set analytics cookies, to see which pages are useful; functional cookies, so that our live chat works; and advertising cookies, to measure our campaigns. We will not set any of those until you say yes, and you can change your mind at any time from the Cookie Settings link in the footer.
Choose the cookies you are happy for us to use. Strictly necessary cookies are always on; everything else is off until you turn it on.
These cookies keep the website working: they hold your security checks, your form submissions and the choice you make here. The site cannot run without them, so they are always on.
These let our live chat open and remember your conversation, and let the newsletter sign-up work. Switch them off and the chat window will not appear.
These tell us which pages are read and how people find us, so that we can improve the site. The reports are about groups of visitors, never about you personally.
These let us measure our advertising and show you our adverts on other websites, such as LinkedIn, Google and Meta. Switch them off and you will still see adverts, but they will be less relevant.