Open-weight vs closed AI models: should your business run its own?

Picture a small physiotherapy clinic that wants AI to sort its patient messages, and the owner keeps hearing that she should "download her own AI" so nothing leaves the building, because it is cheaper and safer. I get versions of this question a lot. Both claims are sometimes true and sometimes badly wrong, and the difference comes down to a few questions you can answer in ten minutes.

This article gives you those questions. First, the two words that confuse everyone.

What "open-weight" and "closed" actually mean

An AI model is, underneath, a huge file of numbers called weights. The weights are what the model learned during training. Whoever holds that file can run the model.

A closed model keeps its weights on the company's servers. You use it through their app or their API (an application programming interface, a way for your software to send text to their servers and get an answer back). Claude, Gemini and ChatGPT's models work this way. You never see the file. You pay per use, usually priced per million tokens (a token is a chunk of text, roughly three quarters of a word in English).

An open-weight model publishes the weights file. Anyone can download it and run it on their own computer or server. Google's Gemma 4 family is a good example: Google released it on 31 March 2026 in sizes it calls E2B, E4B, 31B and 26B A4B, under the Apache 2.0 licence, which allows commercial use.

"Open-weight" is not quite the same as "open source". Open source traditionally means you also get the training code and data, and many open-weight releases share only the finished weights plus a licence. Always read the licence. Some allow anything, some forbid certain commercial uses, and some ask big companies to pay.

What has changed this year

Two things moved at once in 2026, and they pull in opposite directions.

Closed models got much cheaper. On 7 October Anthropic released Claude Haiku 5.5 at $0.10 per million input tokens and $0.50 per million output tokens for prompts up to 100,000 tokens. Its previous small model, Haiku 4.5, cost $1 and $5. Google's new Gemini 4 Argon, announced on 13 October, has an introductory API price of $2 in and $10 out per million tokens until 30 November, then $4 and $20.

Open models got much bigger and much better. Mistral announced Mistral Large 4 on 6 October: 1 trillion parameters in total, 52 billion active at a time, available now as a preview through Mistral's API, with the weights promised for the end of October. (A parameter is one of those learned numbers; "active" means how many the model uses for each word it writes, which drives how fast and how expensive it is to run.)

So the cheap end of closed AI is now very cheap, and the best open models are now so large that running them yourself means data-centre hardware. The open models that fit on a normal computer are the small ones, and that is where a small business should look.

What it really costs to run your own

Here is a worked example with real numbers you can redo for your own business.

Say our clinic gets 10,000 patient messages a month and wants each one summarised and tagged ("booking", "billing", "clinical question"). Assume each message plus instructions is about 1,000 tokens in, and each summary is about 300 tokens out.

At this volume the closed model costs less than a coffee each month. The "free" open model costs a machine, some setup time and someone's attention when it breaks. Running your own does not save money for a small business at normal volumes, and the price cuts this month made that even more true.

How big a machine? Google's own Gemma documentation gives the memory needed just to load the weights. At full 16-bit precision it ranges from about 11.4 GB for the smallest Gemma 4 model to about 69.9 GB for the 31B model. With 4-bit quantization (a compression trick that stores each number with fewer bits, trading a little quality for a lot of memory) that drops to roughly 2.9 GB to 17.5 GB. Google says the E2B and E4B models can run offline on phones and small boards like a Raspberry Pi, and the larger ones are built for consumer graphics cards.

So a recent laptop can run a small open model. A trillion-parameter model like Mistral Large 4 cannot run on a laptop.

The three cases where running your own makes sense

After you strip away the hype, I see three honest reasons to choose open weights.

1. The data must not leave your building

Some data has rules attached: health records, legal files, client data under contract, anything a regulator cares about. Closed providers offer strong privacy terms, and some offer zero data retention, but for some contracts "it never left our machine" is the only answer that ends the conversation. A small open model on a local computer gives you that.

2. You need it to work offline or at the edge

A shop till, a field device, a kiosk with patchy internet. If the AI must answer when the connection is down, it has to live on the device. This is exactly what the small Gemma 4 sizes are designed for.

3. You need to control the model itself

Closed models change. Companies retire old versions and update behaviour. If you have built a process that depends on a model behaving exactly the same for years, owning the weights means nobody can switch it off. You can also fine-tune an open model (train it a little more on your own examples) in ways closed providers may not allow.

If none of these three apply, use a closed model through an app or API and spend your time on the work itself.

When closed is the better choice

For most small businesses, most of the time, closed models win on three counts:

• Quality per dollar. The strongest models today are closed, and the cheap closed tiers now cost cents for real workloads.
• No maintenance. Security patches, uptime and upgrades are someone else's job.
• Features around the model. Search, file handling, image understanding and tools come built in.

The trade is that you depend on a company's prices, rules and servers. That is the same trade you make with email or accounting software, and most businesses accept it.

A quick way to decide

Answer these four questions in order. Stop at the first "yes".

1. Is there a legal or contractual reason the data cannot go to an outside company? If yes, test a small open model locally.
2. Must it work without internet? If yes, test a small open model on the device.
3. Will this process need the exact same model behaviour for years, or custom training you cannot get from a provider? If yes, consider open weights and budget for a technical person.
4. None of the above? Use a closed model. Start with the cheapest tier and move up only if the results are not good enough.

Run the clinic from the start of this article through these questions. Clinical notes probably hit question 1. Marketing emails and booking replies hit none of them. The sensible setup is a closed model for marketing and bookings, and a small local model tested for clinical summaries. I expect most businesses to end up with a split like that.