Running a model on your own hardware was supposed to be a phase. The cloud had won, inference was somebody else’s problem, and local models were a hobby for people with spare graphics cards. That is not how it went. The clearest signal came from the least likely direction: OpenAI, the company that made closed weights the industry default, published a model anyone can download and run on a laptop.
Did local AI come back because hosted AI got expensive?
No, and this is where most explanations go wrong. Hosted inference got dramatically cheaper over the same period. Anthropic’s published pricing puts its small tier at $1 per million input tokens, and the current flagship at a third of what the retired one cost. Running a model yourself is not the frugal option.
Once cost is ruled out, the actual causes become visible. Three things changed at roughly the same time: open weights got genuinely good, the memory required to run them collapsed, and the licences stopped being complicated.
The closed labs started publishing open weights
OpenAI’s gpt-oss-20b is the single most telling artefact in this story. Its model card describes 21 billion parameters with 3.6 billion active, released under Apache 2.0, and states its MXFP4 quantisation lets the model run within 16GB of memory. The card explicitly frames it as being for local or specialised use cases.
Read that as a strategic admission rather than a gift. A company whose entire commercial position rests on hosted access does not publish downloadable weights unless local deployment has become a market it cannot afford to concede. The licence choice reinforces it: Apache 2.0 means no copyleft obligations and no patent risk, which is the licence you pick when you want enterprises to actually use the thing.
When the most closed lab in the industry ships open weights that run on a laptop, the argument is already over. That is a concession, not a contribution.
Capability per gigabyte improved faster than model size grew
The second shift is less discussed and probably more important. Small models got dramatically better at their size, so the memory needed for a genuinely useful model fell rather than rose.
Qwen3:4b is the clearest example. As listed in Ollama’s library it downloads at 2.5GB and carries a 256K context window. A few years ago, that combination did not exist at any size: long context was a frontier feature reserved for the largest hosted models. Now it fits on a machine with 8GB of memory, alongside a browser.
Gemma4:12b at 7.6GB makes the same point at the next tier up, offering a 256K window and image input on hardware most people already own. The practical steps are covered in our guide to running an AI model on your own laptop, and the setup genuinely takes about ten minutes.
The licences stopped being a research project
Early open models arrived under bespoke licences with usage caps, field-of-use restrictions and acceptable-use clauses that a small company could not evaluate without a lawyer. That friction quietly killed a great deal of adoption, because the model being free mattered less than nobody being able to say whether it was usable.
Apache 2.0 removed that. It is a licence procurement departments already understand, and gpt-oss ships under it, as does Alibaba’s Wan 2.2 for video. The same shift is visible in adjacent categories: the practical options in free AI video generation are now the open-weight ones, for exactly the same reason.
What local actually buys you
Four things, none of which is money. Being clear about them prevents the disappointment of expecting a laptop to match a frontier model.
- Confidentiality that is structural, not contractual. A hosted provider can promise not to train on your data. A local model cannot send it anywhere. For legal, medical and client work that difference is the whole argument.
- Permanence. A downloaded model cannot be deprecated, repriced, rate-limited, or have its behaviour changed underneath a product that depends on it.
- Predictable latency. No queue, no rate limit, no outage. Slower on average, but never suddenly unavailable.
- Freedom to modify. Fine-tuning, quantising, and inspecting behaviour are all possible with weights on disk and impossible through an API.
Where the story is oversold
The gap on hard reasoning has not closed and is unlikely to. A 20B model with 3.6B active parameters is not competing with a frontier system on multi-step logic, obscure factual recall, or long autonomous tasks. Anyone claiming parity is comparing benchmarks rather than experience.
There is also a hardware floor that no amount of optimisation removes. LM Studio recommends 16GB of RAM as a working minimum, and video generation is stricter still, with Wan 2.2’s practical variant asking for 24GB of VRAM. Local AI is free at the point of use and not free at the point of purchase.
The honest framing is complementary rather than competitive. Local handles the routine, private, high-volume work; hosted handles the hard problems. Most people who adopt local models end up using both, which is the outcome the pricing pages were never designed to describe.
The bottom line
Local AI came back because the thing you can download became good enough to be worth the disk space, not because the thing you rent became expensive. Open weights from a closed lab, a 256K context window in 2.5GB, and a licence that fits on one line are what moved the argument.
Treat it as a second tool rather than a replacement. The work that is confidential, repetitive, or needs to keep functioning regardless of somebody else’s roadmap belongs on your own machine. The rest can stay in the cloud, where it is cheaper than it has ever been.
Frequently asked questions
Why would anyone run an AI model locally when hosted is cheap?
For confidentiality and permanence rather than cost. A local model physically cannot transmit your data, and it cannot be deprecated, repriced or rate-limited. Hosted inference is genuinely cheaper for most workloads, so cost is not the reason.
Did OpenAI really release an open model?
Yes. gpt-oss-20b is published under Apache 2.0 with 21 billion parameters, of which 3.6 billion are active, and its model card states the quantised version runs within 16GB of memory. It is explicitly described as being for local or specialised use.
Are local AI models as capable as ChatGPT or Claude?
Not on hard reasoning, long autonomous tasks or obscure factual recall, and that gap is unlikely to close. On summarising, rewriting, extraction and document questions, a good local model is close enough that most people would not notice.
What does Apache 2.0 mean for an AI model?
Commercial use is permitted outright, with no copyleft obligations and no patent risk. It is a licence legal teams already recognise, which is why its adoption by open-weight models removed a large amount of friction that bespoke AI licences had created.
What hardware do you need for local AI?
16GB of RAM is the practical working minimum for text models, which is what LM Studio recommends. Small models run in 8GB. Video generation is far more demanding, with open video models typically asking for 24GB of VRAM.
