I still want the expensive model in the chair. I do not want it typing a hundred pytest functions. That work is often bounded and cheap to reject if it is wrong, making it a useful candidate for a small language model (SLM) running on my own hardware. The trick is getting Cursor, Copilot, or Claude to use that model without publishing its Ollama endpoint to the internet.
slm-setup is the setup I am using for that split. The premium agent stays the planner and the reviewer. A local process on the workstation talks to Ollama — an open-source server that runs language models on your own hardware. The local model never becomes “the model in the picker” — the IDE dropdown where you choose GPT or Claude as your chat model. It is a handful of tools the premium agent calls.
The thing that does not work
Cursor can override an OpenAI-compatible base URL, but those requests are assembled on Cursor’s servers. A workstation localhost or home-LAN Ollama URL is therefore outside the default request path; Cursor staff recommend a publicly reachable HTTPS endpoint for that configuration [1] [2] [3]. I do not want to expose Ollama publicly. The override is also shared by the OpenAI-model slot rather than configured per model, so it conflicts with keeping premium OpenAI-family models available for planning [4].
The same boundary applies to default vendor-hosted coding agents: Copilot’s cloud agent runs in GitHub Actions, while Anthropic-hosted Claude Code sessions run in Anthropic-managed virtual machines [5] [7] [8]. Enterprise self-hosted runners and organization-managed Claude environments are exceptions [6] [7], but they are separate designs. This home-lab bridge targets the machine in front of you: desktop Cursor, Copilot agent mode in the IDE, and local Claude Code.
Who does what
If the shape of the answer is obvious — tests, a rename, a bounded refactor, a summary, a first-pass review — I ask the local tools. If the problem is architectural, cross-system, or security-sensitive, I keep it on the premium model. If the local answer is thin or wrong, the premium model edits it. I expect to spend some premium tokens deciding and reviewing. I am trying not to spend them emitting the artifact.
you + premium agent
│ plan, pick files, review
▼
local-coding-slm (stdio, on the workstation)
│ HTTP to localhost or an SSH local forward
▼
Ollama
fast model stronger model
(everyday coding) (harder local work)The IDE and the MCP server live on the workstation. Ollama can run on that same machine or, after the planned two-machine test, on a second private host reached through SSH local forwarding — an encrypted connection that makes the remote Ollama service appear local. Machine-specific connection details never go in git.
What the local model is for
One server name: local-coding-slm. The MCP process does not run shell commands, write files, or open a network listener; it makes outbound HTTP calls to the configured Ollama API. It returns text — code, a diff, or markdown — and I decide whether to apply it. I send a few files, not the whole repository.
The tools have deliberately narrow responsibilities: write code, refactor, generate tests, explain, review, and check that Ollama is up. Each one has a short fixed system prompt. The test tool is told to write tests only. If the request is ambiguous, it should ask instead of inventing production changes.
The starter pair is a fast everyday model (qwen3.5:9b) and a stronger coding model (devstral-small-2). Use fast by default and escalate only when its answer is not good enough. The documented starting configuration uses 16K context — a working window of roughly 16,000 tokens; the stronger model may split work between CPU and GPU when GPU memory is tight [12]. Increase context only after measuring memory use on your own hardware.
What is tested now
Thirty-six mocked unit tests currently pass. Automated live integration checks also reach a real loopback-only Ollama — bound to 127.0.0.1 for access from this machine only — through the stdio MCP server. In a separate manual Cursor check, Cursor invoked the MCP tools and the returned strong-model test file passed both tests when executed [12]. This validates the current same-machine integration, not a complete IDE file-application workflow or broad model reliability.
What is in the repo
The public repo is the spec plus a running server, not a sketch. The server talks to the IDE over stdio — standard input and output, so it opens no network port of its own — and the same-machine path has been exercised against a real local Ollama runtime [12]. You also get a wrapper that starts the server, reads settings from the environment (commonly loaded from a gitignored .env), and templates for Cursor, VS Code Copilot, and Claude Code. Machine-specific values are never committed.
Treat local output as untrusted. Apply it, trim it, or throw it away. The premium agent can access workspace files and sends selected context through the vendor’s service; the SLM sees only the snippets passed in the tool call. There is no automatic classifier deciding which model to call. The project instructions keep the rule explicit: bounded mechanical work can go to the local tools; ambiguous, architectural, or security-sensitive work stays with the premium model.
127.0.0.1:11434 and create a workstation-only forward: ssh -N -T -o ExitOnForwardFailure=yes -L 127.0.0.1:11436:127.0.0.1:11434 user@<inference-host>. Then point MCP at http://127.0.0.1:11436. A private-interface bind is only a firewall-restricted fallback. The current dated repository scan found no detected leaks or GitHub secret-scanning alerts, while noting that a clean scan is not proof of absence [13].
If you already pay for a premium coding agent and have a private GPU, this is the shape to start from. Clone the repo, set OLLAMA_BASE_URL in the gitignored .env file, and measure whether the small model is good enough for the bounded work you actually do.
The host side of this has a roadmap now: an AMD Halo-class box as a later private Ollama host, with the C4 views that show what changes and what does not. That is the follow-up post, Same MCP, next host — Halo on the slm-setup roadmap.
References
- Cursor Docs — Custom API keys: “Your API key … is sent to our backend with every request because all requests are routed through Cursor’s servers for final prompt building.” Also notes custom keys apply to chat models only; Tab completion stays on Cursor’s models.
- Cursor Forum (staff reply, Feb 2026) — Connecting local AI server to Cursor does not work: “All BYOK requests go through Cursor’s servers to build prompts, so localhost or local network addresses won’t work because the server can’t reach them. You’ll need to expose your Ollama instance as a public HTTPS endpoint using something like ngrok or Cloudflare Tunnel.”
- Cursor Forum (staff reply, Mar 2026) — How to use Cursor directly with only my model API?: “There is currently no option to have Cursor communicate directly with your own server without going through Cursor’s backend.” See also staff confirmation that a full bypass is an architectural limitation: “Prompt building, context retrieval, and Cursor Tab and Agent run on our side.”
- Cursor Forum (staff reply) — routing explanation: with a custom key plus override, “requests for OpenAI-family models (anything that’s not claude-* and not gemini-*) go to your custom endpoint”; the base URL is a single global setting, not per-model.
- GitHub Docs — About Copilot cloud agent: the coding agent “has access to its own ephemeral development environment, powered by GitHub Actions,” and is “distinct from the ‘agent mode’ feature available in your IDE,” which “makes autonomous edits directly in your local development environment.”
- GitHub Docs — Customize the agent environment: self-hosted runners can give Copilot access to internal network resources. This is an enterprise exception to GitHub’s default hosted environment. See also the agent firewall documentation: the cloud agent’s internet access is restricted inside the GitHub Actions environment.
- Anthropic — Claude Code on the web: Anthropic-hosted sessions run in isolated Anthropic-managed VMs; organizations can also configure self-hosted cloud environments.
- Anthropic — Configure cloud environments: all outbound traffic from cloud sessions passes through Anthropic’s network proxy with allowlist levels (None / Trusted / Custom / Full).
- Ollama — FAQ and API authentication: Ollama binds to
127.0.0.1:11434by default, and no authentication is required for its local API. - SentinelLABS + Censys — Silent Brothers (Jan 2026): joint 293-day internet scan; 175,108 unique exposed Ollama hosts across 130 countries, 7.23M observations, 48% advertising tool-calling. Censys’s earlier single-day snapshot (Ollama Drama) found 10.6K instances, 1.5K directly promptable.
- LeakIX — 12,000 Ollama instances exposed (Feb 2026): 12,269 unauthenticated instances, roughly 1,000 running versions vulnerable to CVE-2024-37032 (“Probllama”), an unauthenticated path-traversal-to-RCE chain.
- slm-setup — Local acceptance results — 2026-09-06: reproducible commands, hardware details, unit results, live same-machine MCP-to-Ollama checks, and a direct Cursor invocation with executed generated tests.
- slm-setup — Repository security scan — 2026-09-06: the refreshed Gitleaks scan covered repository commit
277f6bdand requested all refs; GitHub returned no secret-scanning alerts; committed-tree and loopback deployment checks passed. The report gives exact scan counts and limits.
Web references were checked on September 6, 2026.
Source: github.com/jmjava/slm-setup — start with spec.md
git clone https://github.com/jmjava/slm-setup.git
