Local coding SLM — premium agents, private GPU, one MCP bridge

Keep the premium subscription. Stop spending it on boilerplate. slm-setup is a public spec for a local coding SLM behind a single MCP bridge. Cursor, GitHub Copilot, and Claude Code stay the planners and reviewers. A private GPU host running Ollama does the bounded, mechanical generation.

This is not a cloud model proxy and not a guide for putting Ollama on the internet. Phase 1 is the architecture, the routing rules, and public-safe client templates. The MCP server itself is the next implementation slice.

The split

Task class	Owner
Planning, architecture, ambiguous debugging, multi-system diagnosis	Premium agent
Boilerplate, tests, mechanical refactors, summaries, explanations	Local SLM via MCP
Local answer looks wrong or incomplete	Premium agent reviews / retries

The goal is not to replace Cursor Ultra, Copilot, or Claude. The goal is to stop burning those tokens on thousands of lines of tests and renames that a 9B–24B quantized model can emit. OpenRouter and other extra routers are out of scope. Premium models come from the agent product itself.

Why MCP, not “Override OpenAI Base URL”

Cursor can point at a custom OpenAI-compatible endpoint. Those requests are assembled on Cursor’s servers. A private LAN or localhost Ollama URL is therefore unreachable unless you publish it as a public HTTPS endpoint — which this spec refuses to do.

Cursor also has a single OpenAI base-URL override. Pointing it at Ollama fights with the included premium models. So the local model is not another entry in the model picker. It is a set of tools.

A local MCP process on the workstation can reach the LAN. Cursor’s cloud path cannot. Same rule for Copilot’s GitHub-hosted agent and Claude Code remote sessions. This bridge is for desktop / local CLI only.

Architecture

Cursor / Copilot / Claude Code
        │  premium agent plans + reviews
        ▼
local-coding-slm MCP (stdio on the workstation)
        │  HTTP, private LAN or localhost
        ▼
Ollama :11434
   fast SLM          strong SLM
 (everyday coding)  (harder coding)

The workstation hosts the IDE and the MCP server. The inference host hosts the GPU and Ollama. They can be the same machine. They can be two machines on one private network. The MCP server only needs an HTTP URL — and that URL lives in the environment, never in git.

Hardware and starter models

VRAM is the scarce resource. A 16 GB NVIDIA GPU is enough for the starter pair. Extra system RAM can load a larger model; it does not make it feel fast for interactive coding. Start context at 16K–32K tokens even if a model advertises 256K.

Role	Suggested Ollama tag	Why
Fast	qwen3.5:9b	Everyday coding; leaves VRAM for context on 16 GB
Strong	devstral-small-2	Software-engineering / multi-file work; near the 16 GB edge

Install both. Default to fast. Escalate to strong only when the premium agent decides the task needs it. Confirm current tags and sizes on ollama.com/library before pulling.

The tool contract

One server name: local-coding-slm. Transport is stdio on the workstation. Tools return plain text — generated code, a unified diff, or markdown. The premium agent decides whether to apply edits. Payloads stay small: send only the files the SLM needs, not the repo.

Tool	Purpose
local_code	New code for a well-specified unit of work
local_refactor	Mechanical, localized rewrite
local_generate_tests	Unit / integration test bodies
local_explain	Explain a snippet or flow
local_review	Cheap first-pass review
local_status	Health of Ollama + listed models

Each tool gets a short, fixed system prompt on the server. Example for tests: generate tests only, match the hinted framework, do not invent production changes, and ask up to three clarifying questions instead of guessing.

The server never executes shell commands, never writes files, and never opens ports other than the configured Ollama URL.

Routing (phase 1 is a paragraph, not a classifier)

Mechanical / repetitive / well-specified  →  local_* tools (fast, then strong)
Ambiguous / architectural / multi-system  →  premium model only
Local answer incomplete or wrong          →  premium model reviews

Delegate when the output shape is obvious, the needed context fits in a few files, and a wrong answer is cheap to reject. Do not delegate incident debugging, broad refactors with unclear invariants, security-sensitive code without premium review, or anything that needs live repo tools the SLM does not have.

The premium model will still spend tokens deciding and reviewing. That is expected. Savings come from not using it to emit the artifact.

One server, three front ends

Committed examples live under examples/ and use interpolation. Real URLs stay in .env or a prompt.

• Cursor — .cursor/mcp.json or ~/.cursor/mcp.json. Leave Ultra / included models unchanged. Do not set Override OpenAI Base URL to Ollama. Cloud Agents cannot see a private LAN host.
• Copilot IDE Agent — .vscode/mcp.json with an inputs prompt for the Ollama URL so a LAN address is never committed. Copilot’s cloud agent on GitHub runners is unsupported for this server.
• Claude Code local — project .mcp.json with ${VAR:-default} expansion. Cloud / remote sessions cannot reach the private GPU.

The same routing paragraph goes in Cursor rules, Copilot instructions, or CLAUDE.md: when the task is mechanical, call the local tools first, prefer model=fast, review before applying, and never send secrets or .env files.

Inference host, then stop

Phase 1 of the rollout is Ollama only. Install it on the GPU machine (keep the current OS; dual-boot is not required). Pull the fast model, confirm GPU placement with ollama ps, optionally bind OLLAMA_HOST=0.0.0.0:11434 for a two-machine LAN, firewall 11434 on the private profile only, and curl /api/tags plus /api/chat from the workstation.

Then stop. Do not build routing until the fast model feels usable. Phase 2 is the stdio MCP server. Phase 3 is measurement — local success rate, latency, escalation rate, rough token savings — and only then maybe an automatic classifier.

No public port forward for 11434. No ngrok, no Cloudflare Tunnel so Cursor can treat Ollama as a model provider. Bind the LAN, firewall the private subnet, keep real hostnames out of git.

What landed, what did not

Merged in jmjava/slm-setup: spec.md, a short README, .env.example, and Cursor / VS Code / Claude MCP templates with placeholders only. The repo is safe to clone. It is not ready-to-run MCP server code yet.

That is the honest Phase 1. Measure whether a 9B local model is good enough for your mechanical work before adding another layer of automation.

Source: github.com/jmjava/slm-setup — start at spec.md
git clone git@github.com:jmjava/slm-setup.git