Run Your First Local Model

Running a model locally should not begin with a grand automation plan. It should begin with custody and evidence. This lesson uses one tested, reproducible path: Ollama as the local runner and `qwen3.5:4b-q4_K_M` as a modest first model. The model build is currently a 3.4 GB download in Ollama's library, and the official Qwen model card lists Apache License 2.0. Those facts do not make the model universally best. They make the first experiment named, inspectable, and repeatable. You will inventory the computer, distinguish model-file size from runtime memory, inspect the model card and license, install the runner, pull the exact model, run two fictional branch-operations tests, inspect processor placement and context, and preserve a baseline. Nothing in this lesson connects company information, web search, email, CRM, tools, memory, schedules, or an agent. First prove local inference. Then decide what the system should become. WHAT YOU WILL LEARN - What “local” means—and why it does not automatically make every connected application local. - Which hardware facts to record before selecting a model. - The difference between parameter count, quantized file size, runtime memory, and context memory. - Why a model card and license belong in the implementation record. - How to install and verify Ollama on a supported computer. - How to pull and run the exact `qwen3.5:4b-q4_K_M` model build. - How to test constrained extraction and honest handling of missing information. - How to use `ollama ps` to inspect processor placement and allocated context. - What evidence to preserve before connecting anything consequential. THE CENTRAL IDEA The first local run is proof that a named set of model weights is present, inference is occurring on hardware you control, and the behavior can be observed and tested.