We pulled nemotron-3.5-lightning on a quiet box and watched VRAM like it was a rent invoice. Ollama's post is dated 11 Aug 2026. NVIDIA's 30B open model, about 3B active at inference, sold as a local agent that can hold context, call tools, and keep walking. That is the part that matters if you ship software, not a demo reel.
A dense 30B used to mean a second mortgage on GPUs. A sparse 30B with a small active set means you can talk about a workstation, a small server, or a Linux box you already own. I still want the tokenizer, the license, and the tool-call format in writing before anyone calls it "production."
What we actually wire
The model is the cheap line item. The product is the glue: which documents it may read, which APIs it may call, how you log a bad tool call, where the transcript lives when legal asks. Custom software development services here means that glue, not a chatbot skin.
Simon Willison flagged Qwen 3.8 27B hitting 52 on Artificial Analysis's intelligence index the same week. Small open weights keep closing the "good enough" gap. That is useful. It is also how teams skip tenancy, evals, and an off switch.
If the model runs on your desk, the desk is now part of the threat model. Prompt injection is still injection. A local agent with shell tools is a junior hire with sudo.
What we refuse
We will not drop an Ollama unit file on a laptop and invoice you for "AI transformation." We will not hide a cloud fallback that ships customer text to a third party because the local context window filled up.
We will tell you when the task is still cheaper as an API. Some workflows want a frontier model once a day and a local model for the other 400 calls. That split is an engineering decision. Open weights are a procurement question when the board asks about residency. This post is about the box.
We install the runtime, pin the model digest, and attach it to the product you already run. If that is the job, request a quote.
I do not know yet whether Lightning stays the default we recommend in six months. The hardware math changed this month. The product math did not.
Sources
- NVIDIA Nemotron 3.5 Lightning — Ollama, 11 Aug 2026
- Qwen 3.8 27B scores 52 on the Artificial Analysis Intelligence Index — Simon Willison, 17 Aug 2026