Open Source AI in 2026: The Real Self-Hosting Break-Even

Open source AI is worth it when your volume is steady, your task is narrow, and your data has to stay inside your own network. It is not worth it because the weights are free. The weights cost nothing. The GPU hours cost $3,953 a month whether a request arrives or not, and that one line settles most of these arguments before the model ever loads. Here is the arithmetic, checked against live prices today.

TL;DR

• Vercel's AI Gateway Production Index, published 13 July 2026, reported open-weight models ran 29 percent of gateway tokens in June on just under 4 percent of spend, up from 11 percent of tokens in April. Volume moved. Money did not.
• Menlo Ventures, in its 9 December 2025 enterprise report, put open source at 11 percent of enterprise LLM API usage, down from 19 percent the year before.
• Stanford HAI's 2026 AI Index has the top closed model ahead of the top open model by 3.3 percent as of March 2026, up from 0.5 percent in August 2024. The gap reopened.
• Together AI lists gpt-oss-120B at $0.15 in and $0.60 out per million tokens against Claude Haiku 4.5 at $1 and $5. Blended, that is about 7.6 times cheaper with no servers to run.
• A dedicated H100 at Together's listed $5.49 per hour is $3,953 a month. To beat that same hosted open model you need roughly 5,800 tokens per second sustained, every second, all month.

Open source AI is three decisions, not one

Almost every bad open source AI decision I have watched came from collapsing three separate questions into one word. They are independent, and only one of them costs real money.

Weights. Can you download the parameter file. That is the only part of "open" that everyone agrees on. Training data and training code almost never ship, which is why the label is contested, and why "open weight" is the more accurate term.

Hosting. Where inference runs. You can call an open model on somebody else's serverless endpoint, rent a dedicated GPU by the hour, or rack your own. These are wildly different cost and operations profiles and they have nothing to do with whether the weights are open.

License. What you are permitted to do. Apache 2.0 and a vendor community agreement are not interchangeable, and the difference shows up in a legal review, not a benchmark.

Say "we're going open source" and you have answered one of these. The other two are where the money and the risk sit.

The one number that explains the market

Open-weight models ran 29 percent of the tokens through Vercel's gateway in June 2026 and under 4 percent of the spend. That ratio, roughly 7 to 1 volume against money, is the entire market in one line.

It means buyers are doing exactly the right thing. The cheap, high-volume, low-stakes work (classify this ticket, extract these fields, summarize this transcript, route this request) has moved to open models where a token costs a rounding error. The expensive, high-stakes work has stayed on closed frontier models, because being wrong there costs more than the inference ever will.

Menlo Ventures measured the same shape from the money side and got what looks like the opposite answer: open source fell from 19 percent of enterprise LLM API usage in 2024 to 11 percent in 2025. Both are true. Enterprises are spending a larger share of a much bigger budget on frontier models while quietly pushing enormous token volume through cheap open ones. Share of spend and share of tokens are different questions, and confusing them is how people end up arguing past each other about whether open source "won."

The practical read: this is a routing decision, not an allegiance. I wrote the full version of that logic in how to choose an AI model, and the cost side in how to cut your AI API bill.

The self-hosting break-even, with real prices

Here is the calculation almost nobody runs before buying hardware.

Together AI's public price list, today, shows a dedicated NVIDIA HGX H100 at $5.49 per hour and an HGX B200 at $8.99. Take the H100:

$5.49/hr x 24 hrs x 30 days = $3,952.80 per month

That number does not move. It is the same on a quiet Sunday as it is during a traffic spike. Now put it against three alternatives, using a 3:1 input-to-output ratio, which is typical for retrieval-heavy production work:

Convert those to a rate you can feel. Divide by 30 days and 86,400 seconds:

Read those three bars carefully, because they say something counterintuitive. Self-hosting almost never beats a hosted open model on price. 5,809 tokens per second sustained every second of every day is a serious production system, and if you had that you would not be reading a blog post about whether to try open source AI.

Where self-hosting does compete is against frontier pricing. 381 tokens per second is roughly a dozen concurrent streaming responses. Plenty of real products clear that during business hours. The trap is that "during business hours" is about a third of the month, so your effective cost triples and you are back above the line.

And the cheapest move on this whole page is not hardware at all. Moving high-volume work from Claude Haiku 4.5 to hosted gpt-oss-120B takes the blended rate from $2.00 to $0.26 per million tokens with zero operational change. gpt-oss-20B at $0.05 in and $0.20 out is $0.09 blended, about 23 times cheaper than Haiku, and it will handle classification and extraction fine. That is the win most teams are actually reaching for when they say open source.

What self-hosting actually buys you

Rent the GPU anyway when one of these is true, and be honest that you are buying it with money rather than saving money:

1. The data cannot leave. Health records, legal discovery, anything under a contract that names your network boundary. This is the argument that survives the arithmetic. Note that it only works if you self-host. Calling an open model on a third-party endpoint puts your prompts on somebody else's servers exactly like a closed API does.
2. The model must not change under you. Hosted models get deprecated and updated. A weights file on your disk does not. If you have spent three months tuning prompts against specific behavior, freezing it has real value. I covered the deployment shape of this in how to deploy an AI agent.
3. You need a fine-tune you own. Adapting weights on your own data and keeping the result is only possible with open weights, and for narrow classification tasks a small fine-tuned model routinely beats a large general one.
4. Latency is the product. Dedicated capacity has no queue and no shared rate limit. If you are inside a voice loop or a keystroke-latency feature, that predictability is worth paying for.

What it costs, in things that are not on the invoice: someone owns quantization, batching, KV cache tuning, load, failover, and the 3am page. Budget an engineer's attention, not just an instance.

The license is not a detail

Two models both described as open source, two very different legal positions.

gpt-oss-120b on Hugging Face ships under Apache 2.0. That is an ordinary permissive software license with a patent grant. Nothing to negotiate. It is 117 billion total parameters with 5.1 billion active per token, and it fits on a single 80GB GPU because the mixture-of-experts weights use MXFP4 quantization, which is why it shows up in so many self-hosting plans.

Llama 4 ships under the Llama 4 Community License Agreement, which is Meta's own document. It grants a royalty-free worldwide license, then attaches conditions Apache 2.0 does not have: you must prominently display "Built with Llama," any derived model name has to begin with "Llama," you must retain Meta's copyright notice and follow its brand guidelines, and if your product had more than 700 million monthly active users on the release date you have to go ask Meta for a separate license.

None of that is unreasonable. It is also not Apache 2.0, and a five minute read of the actual license file before you build on a model is the cheapest risk reduction available to you. Check it before the pilot, not after the product ships.

The setup I'd run

If I were starting this decision today, in order:

1. Write the evaluation first. Fifty real inputs from your product with scored expected outputs. Everything below is unanswerable without it, and a leaderboard is not a substitute. My daily driver stack covers the tools I keep in the loop for this.
2. Run three hosted models against it. One cheap open (gpt-oss-20B), one strong open (gpt-oss-120B or a current DeepSeek or Qwen route), one frontier closed. Same prompts, same evaluation, one afternoon. You now have a real quality-versus-price curve for your task instead of somebody else's.
3. Route, do not migrate. Send the volume tasks to the cheapest model that passes and keep the judgment tasks on the frontier. Put both behind one interface so switching is a config change, which is the whole point of building the API wrapper properly.
4. Track utilization for 30 days. Actual tokens per second, including nights and weekends. Now the break-even table above answers itself.
5. Only then consider hardware. And if you do, rent by the hour before you buy anything, because the second month of real traffic will tell you more than any projection.

The bottom line

Open source AI in 2026 is not a cost-saving strategy on its own. It is three decisions wearing one name, and only one of them (where inference runs) actually moves your bill. The market has already voted with the shape Vercel measured: 29 percent of tokens, 4 percent of spend. Cheap open models absorb volume. Frontier closed models keep the work where being wrong is expensive. Self-hosting is a control purchase that pays for itself only at utilization most teams never reach, and the honest test is whether you can name the compliance requirement or the sustained tokens per second that justifies it. If you cannot, rent the endpoint and spend the saved month building the product.

Run the numbers on your own workload this week: pull last month's token counts, blend them 3:1, and put them against the table above. Then grab The AI Daily Driver Stack for the tools I actually keep in rotation, and join the newsletter for the build breakdowns I only send there.