In this hands-on tutorial, Leon van Zyl pushes back on the common claim that free local models are useless for real coding. His core thesis: it's not the model, it's the harness. The reason local models fall apart when you plug them into a heavy agent like Claude Code is that the harness eats up the tiny context window these models have before you even send your first message. His fix is to pair a lean harness, the Pi Agent SDK, with local models served by Ollama, giving you a fast, free coding agent that runs entirely on your own hardware.
Van Zyl opens with the familiar failure story: you download a local model, plug it into Claude Code, and everything falls apart. His point is that the model is rarely the culprit. The moment Claude Code opens, it loads about 20,000 tokens into the context window, an Anthropic system prompt plus the internal tools for the harness, and that figure does not even include MCP or skills. The user has no control over this overhead.
Because these models run on your own hardware and get loaded into VRAM, they can only hold so many tokens, usually around 120,000 to 130,000, and up to roughly 200,000 on larger setups. If a harness automatically spends 20,000 to 30,000 tokens, you quickly reach what he calls the danger zone. Every model, including the flagships, starts losing intelligence and quality once you hit a certain threshold of the context window, typically the 50 to 70 percent mark. The lesson: keep the instructions in a session as tight and to the point as possible and never come close to exceeding your context window. For a free model with only about 130,000 tokens, you cannot afford to lose 30 percent to harness overhead.
The second problem with free models is that they get confused and overwhelmed easily. They simply do not have the tool-calling and reasoning capabilities of flagship models. Since harnesses like Claude Code contain a lot of tools, these smaller agents get overwhelmed and do not know what to do.
Van Zyl's recommendation is to run free local models inside a lean harness such as the Pi Agent SDK (or OpenCode, which he covered in a previous video). These harnesses inject only the minimum tools required for a coding agent. For the Pi Agent SDK specifically, that means very few tools: it only lets the agent look up and make changes to files in a specific folder. Natively it does not support MCP, skills, web search, or other tools that might blow the context window. The philosophy is a fast, lean base where you decide what to bolt on afterward, and you can add those extra capabilities through the plugin marketplace.
Go to pi.dev. To install the agent, select NPM, copy the install command, then open command prompt, PowerShell, or your terminal and run it. Once done, run the command again to check it works; at that point you should see the chat interface.
To run a model locally, install Ollama. Go to ollama.com, click download, select your operating system, and install. After installing, open your command prompt or terminal and run ollama; if you get the menu, everything worked. Then download a model from the models list. Van Zyl suggests asking ChatGPT or Claude to research which models your machine can actually run.
Van Zyl says he is a big fan of the model he references as "23.6" (as heard in the transcript) and considers it the base coding model you can run on your hardware right now. It comes in two flavors: a 27 billion parameter version needing about 16 GB of VRAM, and a 35 billion parameter version needing about 24 GB of VRAM. To use it, copy the model name, then in command prompt run ollama pull followed by the model name, and once downloaded run ollama run followed by the model name.
Van Zyl's argument is that the disappointing results people get from local coding models are mostly a harness problem, not a model problem. By pairing a lean harness like the Pi Agent SDK with Ollama-served local models and keeping instructions tight, you can build software reliably for free on your own hardware. Start minimal, protect your limited context window, and add capabilities like MCP and skills only when you need them.