The GPU can move. The bridge should not. The first slm-setup post described a split: a premium cloud model plans and reviews, while a small language model (SLM) on private hardware handles bounded generation through MCP tools. The private model runs under Ollama, an open-source server for running language models locally. “Halo” here means a PC built around AMD’s Strix Halo design, which can share up to 128GB of unified memory between the CPU and integrated graphics. The question is whether that memory makes Halo a worthwhile later Ollama host — not whether to build a second coding system.
Two C4 diagrams make the roadmap concrete. The context view shows who talks to what; the deployment view shows which machine runs each part. The software boundary stays fixed while the machine hosting Ollama changes.
What stays fixed
Context. The private lab uses a premium model for planning while Ollama remains private. Cloud-hosted agents are outside this home-lab design.
Cursor, Copilot, or Claude remains the planner. The private model is exposed through MCP, the Model Context Protocol: an open standard that lets the desktop IDE call a local program through named tools. Default vendor-hosted agents run away from the workstation [16] [17] [18]; enterprise private-connectivity and self-hosted configurations are possible, but they are separate designs. This home-lab bridge runs on the workstation, where it can reach private hardware without making Ollama public.
The workstation keeps the same stdio MCP server, the same six bounded coding tools, and the same environment settings. Stdio means the IDE launches the server and communicates through standard input and output rather than an MCP network port. The server reads the Ollama address from the environment, commonly through a gitignored .env file. Moving Ollama does not change what the IDE calls.
Deployment — one active host
Variants. Only one host is active. Prefer SSH to that host’s localhost. A private-interface bind is optional. A public bind is not.
The inference host is simply the machine running Ollama. The roadmap changes that host in three controlled stages:
- Now: Ollama runs on the workstation, and the desktop-to-Ollama path is operational [13].
- Next: Ollama moves to a second private GPU host. SSH local forwarding — an encrypted connection that makes the remote service appear local — keeps Ollama bound to that host’s loopback address, which accepts connections only from the host itself. A robust example is
ssh -N -T -o ExitOnForwardFailure=yes -L 127.0.0.1:11436:127.0.0.1:11434 user@<inference-host>; MCP then useshttp://127.0.0.1:11436. - Later: Halo takes the same host slot. Its AMD software backend still needs validation on the actual machine before any performance claim.
The security rule is unchanged: do not make Ollama internet-reachable. A 293-day SentinelLABS–Censys scan measured 175,108 internet-reachable Ollama hosts across 130 countries [5]; LeakIX separately documented 12,269 unauthenticated instances in its February 2026 dataset [6].
Current test status
Thirty-six mocked unit tests currently pass. Automated live integration checks also reach a real loopback-only Ollama through the stdio MCP server. A separate manual Cursor check invoked the MCP tools and executed the strong model’s returned test file successfully [13]. This validates the same-machine integration, not a complete IDE file-application workflow, the planned SSH deployment, broad model reliability, or Halo performance.
What a Halo box costs
The top-end chip considered here is AMD’s Ryzen AI Max+ 395: sixteen CPU cores, Radeon 8060S integrated graphics, and up to 128GB of soldered LPDDR5X unified memory. Framework says as much as 96GB can be graphics-addressable on its 128GB build [1]. That is memory capacity for quantized model weights larger than 16GB — not proof that a particular model, Ollama release, or AMD backend will run well.
Observed September 6, 2026 listings put 128GB systems between $3,449 and $4,349: Framework at $3,449, GMKtec at $3,649.99, and Beelink at $4,349 [1] [2] [3]. Published comparisons place earlier launch-era systems near $2,000 and attribute much of the later increase to soldered-memory pricing [3] [4]. Because that memory cannot be upgraded, capacity is a purchase-time decision. These figures are a dated snapshot, not a quotation.
What would justify buying it
Memory headroom matters only if larger models produce more complete and correct work on representative bounded tasks. Aider’s independent benchmark illustrates the risk: when models extract large methods from real Python projects, completion rates vary sharply and weaker models often skip code [7]. That benchmark covers hosted models, not this Halo proposal, so it supplies a useful test shape rather than a result.
Qwen’s vendor-authored report gives a second reason to compare sizes: within the Qwen2.5-Coder family, its 32-billion-parameter model scores above the 7-billion-parameter version on the published coding benchmarks [8]. Ollama lists the quantized 32B download at 20GB [11], already larger than a 16GB GPU before runtime overhead. Halo has room to evaluate it, but memory fit does not prove a quality gain.
Quality also has to fit the time budget. A direct community benchmark reports 4.7–4.9 output tokens per second for Llama 3.1 70B at 4-bit quantization on Strix Halo [9]. At five tokens per second, a 1,000-token answer takes about 200 seconds before prompt processing. That approaches the bridge’s current 300-second strong-model timeout [14], so a dense 70B model is practical only for bounded synchronous outputs unless the request contract changes. Backend and long-context results also vary materially [10] [12]. Replacing Ollama may require a client change unless the replacement preserves Ollama’s HTTP API.
The purchase criterion is therefore simple: benchmark named models, quantizations, context sizes, and backends on representative work; buy the memory headroom only if the quality gain is worth both the price and the wait. If the current models are already reliable enough, keep the money.
What is in the repo
The public repo provides the runnable MCP server, SSH-first deployment guidance, and dated operational and security evidence [13] [15]. Start with the working same-machine profile. Move to a private host only when that need is real, and consider Halo only after representative work shows that larger models would earn their additional cost. Keep machine-specific values out of git and review every local-model result before applying it.
References
- Framework — Framework Desktop with AMD Ryzen AI Max: Max+ 395 specs (16 cores, Radeon 8060S, LPDDR5X-8000), “up to 96GB of graphics addressable memory,” 128GB configuration at $3,449, sold as pre-order.
- Micro Center — GMKtec EVO-X2 listing: Ryzen AI Max+ 395, 128GB LPDDR5X-8000, 2TB SSD, $3,649.99.
- ComputingForGeeks — Ryzen AI Max+ 395 mini PCs compared (Aug 2026): side-by-side of Framework ($3,449), GMKtec ($3,649.99), Beelink GTR9 Pro ($4,349); documents the June-to-August price doubling and attributes it to DRAM contract pricing on soldered LPDDR5X.
- Liliputing — 128GB Ryzen AI Max+ 395 mini PCs roundup: earlier-2026 price baseline across nine vendors and the note that most of the 128GB can be used as VRAM.
- SentinelLABS + Censys — Silent Brothers (Jan 2026): joint 293-day internet scan measuring 175,108 unique exposed Ollama hosts across 130 countries (7.23M observations); 48% advertised tool-calling capability.
- LeakIX — 12,000 Ollama instances exposed (Feb 2026): 12,269 unauthenticated public instances, ~1,000 vulnerable to CVE-2024-37032, an unauthenticated remote-code-execution chain.
- Aider — Refactoring leaderboard and the benchmark design: 89 large-method extractions from real Python repositories, verified by parsing the output, built to provoke and quantify models eliding code on long outputs. Scores on the published board run from 92.1% at the top down to roughly 21% for the weakest listed configuration.
- Qwen team — Qwen2.5-Coder family report: six sizes trained identically to “verify the effectiveness of scaling”; 32B-Instruct outperforms 7B across the published code benchmarks (92.7 vs 88.4 on HumanEval) and scores 73.7 on Aider code repair, “performing comparably to GPT-4o.”
- ignasivt — Strix Halo Guide: community measurements for Llama 3.1 70B Q4_K_M report 4.7–4.9 output tokens per second across short prompt lengths on a Ryzen AI Max+ 395. This is a direct practitioner benchmark, not vendor certification.
- Strix Halo Wiki — llama.cpp performance: links reproducible backend comparisons and shows that prompt processing, token generation, driver choice, and long-context behavior can differ materially.
- Ollama — qwen2.5-coder library page: the 32b tag is a 20GB download in the default quantization.
- Digital Architects — Ryzen AI Max+ 395 local-LLM field notes: a practitioner report describing material stability and performance differences among Vulkan, ROCm, llama.cpp, and Ollama on Strix Halo. These observations motivate validation on the actual machine.
- slm-setup — Local acceptance results — 2026-09-06: reproducible commands, unit results, live same-machine MCP-to-Ollama checks, direct Cursor invocation, retries, and stated limitations.
- slm-setup — Ollama client implementation: current synchronous request behavior, 300-second strong-model timeout, and 4,096-token output cap.
- slm-setup — Repository security scan — 2026-09-06: refreshed Gitleaks history and committed-tree results at repository commit
277f6bd, GitHub secret-scanning alert state, deployment-safety checks, exact counts, and stated limitations. - Cursor Docs — Cloud Agent security: each Cursor Cloud Agent runs in an isolated cloud virtual machine rather than on the developer’s laptop.
- GitHub Docs — About Copilot cloud agent: the hosted agent runs in an ephemeral GitHub Actions environment, while IDE agent mode edits in the local development environment.
- Anthropic Docs — Claude Code on the web: Anthropic-hosted sessions run in isolated Anthropic-managed virtual machines; organizations can also configure self-hosted cloud environments.
Web references and prices were checked on September 6, 2026. Retail prices and practitioner performance reports are time-sensitive.
Source: github.com/jmjava/slm-setup — spec.md, docs/c4.md, docs/roadmap.md
git clone https://github.com/jmjava/slm-setup.git