Sunday, September 06, 2026

Same MCP, next host — Halo on the slm-setup roadmap

The GPU can move. The bridge should not. The first slm-setup post described a split: a premium cloud model plans and reviews, while a small language model (SLM) on private hardware handles bounded generation through MCP tools. The private model runs under Ollama, an open-source server for running language models locally. “Halo” here means a PC built around AMD’s Strix Halo design, which can share up to 128GB of unified memory between the CPU and integrated graphics. The question is whether that memory makes Halo a worthwhile later Ollama host — not whether to build a second coding system.

Two C4 diagrams make the roadmap concrete. The context view shows who talks to what; the deployment view shows which machine runs each part. The software boundary stays fixed while the machine hosting Ollama changes.

What stays fixed

C4 context: developer and desktop IDE use a private coding lab; premium model APIs plan and review; default cloud-hosted agents are not connected to the private GPU
Context. The private lab uses a premium model for planning while Ollama remains private. Cloud-hosted agents are outside this home-lab design.

Cursor, Copilot, or Claude remains the planner. The private model is exposed through MCP, the Model Context Protocol: an open standard that lets the desktop IDE call a local program through named tools. Default vendor-hosted agents run away from the workstation [16] [17] [18]; enterprise private-connectivity and self-hosted configurations are possible, but they are separate designs. This home-lab bridge runs on the workstation, where it can reach private hardware without making Ollama public.

Local inference is not an air gap. The premium agent can access workspace files and receives the tool results through the vendor’s service. Halo does not hide that selected context from the planner; it keeps the delegated model generation on private hardware.

The workstation keeps the same stdio MCP server, the same six bounded coding tools, and the same environment settings. Stdio means the IDE launches the server and communicates through standard input and output rather than an MCP network port. The server reads the Ollama address from the environment, commonly through a gitignored .env file. Moving Ollama does not change what the IDE calls.

Deployment — one active host

Deployment variants: same-machine Ollama now; optional second private GPU over SSH; later Halo-class AMD with its inference backend still to validate
Variants. Only one host is active. Prefer SSH to that host’s localhost. A private-interface bind is optional. A public bind is not.

The inference host is simply the machine running Ollama. The roadmap changes that host in three controlled stages:

  1. Now: Ollama runs on the workstation, and the desktop-to-Ollama path is operational [13].
  2. Next: Ollama moves to a second private GPU host. SSH local forwarding — an encrypted connection that makes the remote service appear local — keeps Ollama bound to that host’s loopback address, which accepts connections only from the host itself. A robust example is ssh -N -T -o ExitOnForwardFailure=yes -L 127.0.0.1:11436:127.0.0.1:11434 user@<inference-host>; MCP then uses http://127.0.0.1:11436.
  3. Later: Halo takes the same host slot. Its AMD software backend still needs validation on the actual machine before any performance claim.

The security rule is unchanged: do not make Ollama internet-reachable. A 293-day SentinelLABS–Censys scan measured 175,108 internet-reachable Ollama hosts across 130 countries [5]; LeakIX separately documented 12,269 unauthenticated instances in its February 2026 dataset [6].

Current test status

Thirty-six mocked unit tests currently pass. Automated live integration checks also reach a real loopback-only Ollama through the stdio MCP server. A separate manual Cursor check invoked the MCP tools and executed the strong model’s returned test file successfully [13]. This validates the same-machine integration, not a complete IDE file-application workflow, the planned SSH deployment, broad model reliability, or Halo performance.

What a Halo box costs

The top-end chip considered here is AMD’s Ryzen AI Max+ 395: sixteen CPU cores, Radeon 8060S integrated graphics, and up to 128GB of soldered LPDDR5X unified memory. Framework says as much as 96GB can be graphics-addressable on its 128GB build [1]. That is memory capacity for quantized model weights larger than 16GB — not proof that a particular model, Ollama release, or AMD backend will run well.

Observed September 6, 2026 listings put 128GB systems between $3,449 and $4,349: Framework at $3,449, GMKtec at $3,649.99, and Beelink at $4,349 [1] [2] [3]. Published comparisons place earlier launch-era systems near $2,000 and attribute much of the later increase to soldered-memory pricing [3] [4]. Because that memory cannot be upgraded, capacity is a purchase-time decision. These figures are a dated snapshot, not a quotation.

What would justify buying it

Memory headroom matters only if larger models produce more complete and correct work on representative bounded tasks. Aider’s independent benchmark illustrates the risk: when models extract large methods from real Python projects, completion rates vary sharply and weaker models often skip code [7]. That benchmark covers hosted models, not this Halo proposal, so it supplies a useful test shape rather than a result.

Qwen’s vendor-authored report gives a second reason to compare sizes: within the Qwen2.5-Coder family, its 32-billion-parameter model scores above the 7-billion-parameter version on the published coding benchmarks [8]. Ollama lists the quantized 32B download at 20GB [11], already larger than a 16GB GPU before runtime overhead. Halo has room to evaluate it, but memory fit does not prove a quality gain.

Quality also has to fit the time budget. A direct community benchmark reports 4.7–4.9 output tokens per second for Llama 3.1 70B at 4-bit quantization on Strix Halo [9]. At five tokens per second, a 1,000-token answer takes about 200 seconds before prompt processing. That approaches the bridge’s current 300-second strong-model timeout [14], so a dense 70B model is practical only for bounded synchronous outputs unless the request contract changes. Backend and long-context results also vary materially [10] [12]. Replacing Ollama may require a client change unless the replacement preserves Ollama’s HTTP API.

The purchase criterion is therefore simple: benchmark named models, quantizations, context sizes, and backends on representative work; buy the memory headroom only if the quality gain is worth both the price and the wait. If the current models are already reliable enough, keep the money.

What is in the repo

The public repo provides the runnable MCP server, SSH-first deployment guidance, and dated operational and security evidence [13] [15]. Start with the working same-machine profile. Move to a private host only when that need is real, and consider Halo only after representative work shows that larger models would earn their additional cost. Keep machine-specific values out of git and review every local-model result before applying it.

References

  1. Framework — Framework Desktop with AMD Ryzen AI Max: Max+ 395 specs (16 cores, Radeon 8060S, LPDDR5X-8000), “up to 96GB of graphics addressable memory,” 128GB configuration at $3,449, sold as pre-order.
  2. Micro Center — GMKtec EVO-X2 listing: Ryzen AI Max+ 395, 128GB LPDDR5X-8000, 2TB SSD, $3,649.99.
  3. ComputingForGeeks — Ryzen AI Max+ 395 mini PCs compared (Aug 2026): side-by-side of Framework ($3,449), GMKtec ($3,649.99), Beelink GTR9 Pro ($4,349); documents the June-to-August price doubling and attributes it to DRAM contract pricing on soldered LPDDR5X.
  4. Liliputing — 128GB Ryzen AI Max+ 395 mini PCs roundup: earlier-2026 price baseline across nine vendors and the note that most of the 128GB can be used as VRAM.
  5. SentinelLABS + Censys — Silent Brothers (Jan 2026): joint 293-day internet scan measuring 175,108 unique exposed Ollama hosts across 130 countries (7.23M observations); 48% advertised tool-calling capability.
  6. LeakIX — 12,000 Ollama instances exposed (Feb 2026): 12,269 unauthenticated public instances, ~1,000 vulnerable to CVE-2024-37032, an unauthenticated remote-code-execution chain.
  7. Aider — Refactoring leaderboard and the benchmark design: 89 large-method extractions from real Python repositories, verified by parsing the output, built to provoke and quantify models eliding code on long outputs. Scores on the published board run from 92.1% at the top down to roughly 21% for the weakest listed configuration.
  8. Qwen team — Qwen2.5-Coder family report: six sizes trained identically to “verify the effectiveness of scaling”; 32B-Instruct outperforms 7B across the published code benchmarks (92.7 vs 88.4 on HumanEval) and scores 73.7 on Aider code repair, “performing comparably to GPT-4o.”
  9. ignasivt — Strix Halo Guide: community measurements for Llama 3.1 70B Q4_K_M report 4.7–4.9 output tokens per second across short prompt lengths on a Ryzen AI Max+ 395. This is a direct practitioner benchmark, not vendor certification.
  10. Strix Halo Wiki — llama.cpp performance: links reproducible backend comparisons and shows that prompt processing, token generation, driver choice, and long-context behavior can differ materially.
  11. Ollama — qwen2.5-coder library page: the 32b tag is a 20GB download in the default quantization.
  12. Digital Architects — Ryzen AI Max+ 395 local-LLM field notes: a practitioner report describing material stability and performance differences among Vulkan, ROCm, llama.cpp, and Ollama on Strix Halo. These observations motivate validation on the actual machine.
  13. slm-setup — Local acceptance results — 2026-09-06: reproducible commands, unit results, live same-machine MCP-to-Ollama checks, direct Cursor invocation, retries, and stated limitations.
  14. slm-setup — Ollama client implementation: current synchronous request behavior, 300-second strong-model timeout, and 4,096-token output cap.
  15. slm-setup — Repository security scan — 2026-09-06: refreshed Gitleaks history and committed-tree results at repository commit 277f6bd, GitHub secret-scanning alert state, deployment-safety checks, exact counts, and stated limitations.
  16. Cursor Docs — Cloud Agent security: each Cursor Cloud Agent runs in an isolated cloud virtual machine rather than on the developer’s laptop.
  17. GitHub Docs — About Copilot cloud agent: the hosted agent runs in an ephemeral GitHub Actions environment, while IDE agent mode edits in the local development environment.
  18. Anthropic Docs — Claude Code on the web: Anthropic-hosted sessions run in isolated Anthropic-managed virtual machines; organizations can also configure self-hosted cloud environments.

Web references and prices were checked on September 6, 2026. Retail prices and practitioner performance reports are time-sensitive.

Source: github.com/jmjava/slm-setupspec.md, docs/c4.md, docs/roadmap.md
git clone https://github.com/jmjava/slm-setup.git

Tuesday, September 01, 2026

Local coding SLM — premium agents, private GPU, one MCP bridge

I still want the expensive model in the chair. I do not want it typing a hundred pytest functions. That work is often bounded and cheap to reject if it is wrong, making it a useful candidate for a small language model (SLM) running on my own hardware. The trick is getting Cursor, Copilot, or Claude to use that model without publishing its Ollama endpoint to the internet.

slm-setup is the setup I am using for that split. The premium agent stays the planner and the reviewer. A local process on the workstation talks to Ollama — an open-source server that runs language models on your own hardware. The local model never becomes “the model in the picker” — the IDE dropdown where you choose GPT or Claude as your chat model. It is a handful of tools the premium agent calls.

The thing that does not work

Cursor can override an OpenAI-compatible base URL, but those requests are assembled on Cursor’s servers. A workstation localhost or home-LAN Ollama URL is therefore outside the default request path; Cursor staff recommend a publicly reachable HTTPS endpoint for that configuration [1] [2] [3]. I do not want to expose Ollama publicly. The override is also shared by the OpenAI-model slot rather than configured per model, so it conflicts with keeping premium OpenAI-family models available for planning [4].

The same boundary applies to default vendor-hosted coding agents: Copilot’s cloud agent runs in GitHub Actions, while Anthropic-hosted Claude Code sessions run in Anthropic-managed virtual machines [5] [7] [8]. Enterprise self-hosted runners and organization-managed Claude environments are exceptions [6] [7], but they are separate designs. This home-lab bridge targets the machine in front of you: desktop Cursor, Copilot agent mode in the IDE, and local Claude Code.

A process on the workstation can reach a private GPU. The default vendor-hosted path does not share that network. That is the whole reason this is MCP — the Model Context Protocol, an open standard that lets a desktop IDE agent launch a local program and call it as a set of named tools — and not a model-provider hack.

Who does what

If the shape of the answer is obvious — tests, a rename, a bounded refactor, a summary, a first-pass review — I ask the local tools. If the problem is architectural, cross-system, or security-sensitive, I keep it on the premium model. If the local answer is thin or wrong, the premium model edits it. I expect to spend some premium tokens deciding and reviewing. I am trying not to spend them emitting the artifact.

you + premium agent
        │  plan, pick files, review
        ▼
local-coding-slm  (stdio, on the workstation)
        │  HTTP to localhost or an SSH local forward
        ▼
Ollama
   fast model              stronger model
 (everyday coding)      (harder local work)

The IDE and the MCP server live on the workstation. Ollama can run on that same machine or, after the planned two-machine test, on a second private host reached through SSH local forwarding — an encrypted connection that makes the remote Ollama service appear local. Machine-specific connection details never go in git.

What the local model is for

One server name: local-coding-slm. The MCP process does not run shell commands, write files, or open a network listener; it makes outbound HTTP calls to the configured Ollama API. It returns text — code, a diff, or markdown — and I decide whether to apply it. I send a few files, not the whole repository.

The tools have deliberately narrow responsibilities: write code, refactor, generate tests, explain, review, and check that Ollama is up. Each one has a short fixed system prompt. The test tool is told to write tests only. If the request is ambiguous, it should ask instead of inventing production changes.

The starter pair is a fast everyday model (qwen3.5:9b) and a stronger coding model (devstral-small-2). Use fast by default and escalate only when its answer is not good enough. The documented starting configuration uses 16K context — a working window of roughly 16,000 tokens; the stronger model may split work between CPU and GPU when GPU memory is tight [12]. Increase context only after measuring memory use on your own hardware.

What is tested now

Thirty-six mocked unit tests currently pass. Automated live integration checks also reach a real loopback-only Ollama — bound to 127.0.0.1 for access from this machine only — through the stdio MCP server. In a separate manual Cursor check, Cursor invoked the MCP tools and the returned strong-model test file passed both tests when executed [12]. This validates the current same-machine integration, not a complete IDE file-application workflow or broad model reliability.

What is in the repo

The public repo is the spec plus a running server, not a sketch. The server talks to the IDE over stdio — standard input and output, so it opens no network port of its own — and the same-machine path has been exercised against a real local Ollama runtime [12]. You also get a wrapper that starts the server, reads settings from the environment (commonly loaded from a gitignored .env), and templates for Cursor, VS Code Copilot, and Claude Code. Machine-specific values are never committed.

Treat local output as untrusted. Apply it, trim it, or throw it away. The premium agent can access workspace files and sends selected context through the vendor’s service; the SLM sees only the snippets passed in the tool call. There is no automatic classifier deciding which model to call. The project instructions keep the rule explicit: bounded mechanical work can go to the local tools; ambiguous, architectural, or security-sensitive work stays with the premium model.

Do not expose Ollama publicly. No ngrok, Cloudflare Tunnel, or router port-forward that makes the API internet-reachable. Ollama binds to localhost by default, and its local API requires no authentication [9]. A 293-day SentinelLABS–Censys joint scan measured 175,108 unique internet-reachable Ollama hosts across 130 countries [10], and LeakIX found roughly a thousand of the instances it counted still vulnerable to a known unauthenticated remote-code-execution chain [11]. For a second host, keep remote Ollama on 127.0.0.1:11434 and create a workstation-only forward: ssh -N -T -o ExitOnForwardFailure=yes -L 127.0.0.1:11436:127.0.0.1:11434 user@<inference-host>. Then point MCP at http://127.0.0.1:11436. A private-interface bind is only a firewall-restricted fallback. The current dated repository scan found no detected leaks or GitHub secret-scanning alerts, while noting that a clean scan is not proof of absence [13].

If you already pay for a premium coding agent and have a private GPU, this is the shape to start from. Clone the repo, set OLLAMA_BASE_URL in the gitignored .env file, and measure whether the small model is good enough for the bounded work you actually do.

The host side of this has a roadmap now: an AMD Halo-class box as a later private Ollama host, with the C4 views that show what changes and what does not. That is the follow-up post, Same MCP, next host — Halo on the slm-setup roadmap.

References

  1. Cursor Docs — Custom API keys: “Your API key … is sent to our backend with every request because all requests are routed through Cursor’s servers for final prompt building.” Also notes custom keys apply to chat models only; Tab completion stays on Cursor’s models.
  2. Cursor Forum (staff reply, Feb 2026) — Connecting local AI server to Cursor does not work: “All BYOK requests go through Cursor’s servers to build prompts, so localhost or local network addresses won’t work because the server can’t reach them. You’ll need to expose your Ollama instance as a public HTTPS endpoint using something like ngrok or Cloudflare Tunnel.”
  3. Cursor Forum (staff reply, Mar 2026) — How to use Cursor directly with only my model API?: “There is currently no option to have Cursor communicate directly with your own server without going through Cursor’s backend.” See also staff confirmation that a full bypass is an architectural limitation: “Prompt building, context retrieval, and Cursor Tab and Agent run on our side.”
  4. Cursor Forum (staff reply) — routing explanation: with a custom key plus override, “requests for OpenAI-family models (anything that’s not claude-* and not gemini-*) go to your custom endpoint”; the base URL is a single global setting, not per-model.
  5. GitHub Docs — About Copilot cloud agent: the coding agent “has access to its own ephemeral development environment, powered by GitHub Actions,” and is “distinct from the ‘agent mode’ feature available in your IDE,” which “makes autonomous edits directly in your local development environment.”
  6. GitHub Docs — Customize the agent environment: self-hosted runners can give Copilot access to internal network resources. This is an enterprise exception to GitHub’s default hosted environment. See also the agent firewall documentation: the cloud agent’s internet access is restricted inside the GitHub Actions environment.
  7. Anthropic — Claude Code on the web: Anthropic-hosted sessions run in isolated Anthropic-managed VMs; organizations can also configure self-hosted cloud environments.
  8. Anthropic — Configure cloud environments: all outbound traffic from cloud sessions passes through Anthropic’s network proxy with allowlist levels (None / Trusted / Custom / Full).
  9. Ollama — FAQ and API authentication: Ollama binds to 127.0.0.1:11434 by default, and no authentication is required for its local API.
  10. SentinelLABS + Censys — Silent Brothers (Jan 2026): joint 293-day internet scan; 175,108 unique exposed Ollama hosts across 130 countries, 7.23M observations, 48% advertising tool-calling. Censys’s earlier single-day snapshot (Ollama Drama) found 10.6K instances, 1.5K directly promptable.
  11. LeakIX — 12,000 Ollama instances exposed (Feb 2026): 12,269 unauthenticated instances, roughly 1,000 running versions vulnerable to CVE-2024-37032 (“Probllama”), an unauthenticated path-traversal-to-RCE chain.
  12. slm-setup — Local acceptance results — 2026-09-06: reproducible commands, hardware details, unit results, live same-machine MCP-to-Ollama checks, and a direct Cursor invocation with executed generated tests.
  13. slm-setup — Repository security scan — 2026-09-06: the refreshed Gitleaks scan covered repository commit 277f6bd and requested all refs; GitHub returned no secret-scanning alerts; committed-tree and loopback deployment checks passed. The report gives exact scan counts and limits.

Web references were checked on September 6, 2026.

Source: github.com/jmjava/slm-setup — start with spec.md
git clone https://github.com/jmjava/slm-setup.git

Sunday, August 30, 2026

Cursor sessions that survive the chat — tracking work with Obsidian MCP

A Cursor chat is a terrible filing cabinet. It is excellent while it is open. It is gone when you start the next one — or when a Cloud Agent finishes a wave on a different machine. I wanted session memory I can open next week: what we did, which commit we left on, what we decided, and what is still open. That is what obsidian-mcp writes.

Holographic coder connected to floating session notes
The bet: session notes should be files, not a vendor memory API.

The server is local MCP over stdio. Cursor (or Copilot) calls tools. The tools read and write ordinary Markdown in an Obsidian vault. Obsidian does not need to be running. There is no community plugin and no hosted memory database. If I can open the file in a text editor, the memory is real.

What I actually keep

Each project gets one folder under AI Memory/Projects/<slug>/:

Project State.md          # current objective, in-progress, next steps
Sessions/YYYY-MM-DD.md    # timestamped sections for that day
Decisions/YYYY-MM-DD-*.md # one file per durable choice

Project State is the hot brief — what this repo is for right now. It is replaced when status changes. It is not a diary.

Sessions are the diary. capture_work_session appends a timestamped section to today. If I pass the repo path, the note records branch, short SHA, dirty flag, and a short file list. Full diffs never go in. I do not want a second copy of git.

Decisions are the “why.” Architecture choices that the next agent should not re-litigate. Same slug on the same day gets -2, never an overwrite.

This morning’s vault looks like this — a real session note for embabel-v1-learning, not a mock:

Obsidian vault showing AI Memory projects and the 2026-08-30 embabel-v1-learning session
Obsidian on the same files the MCP server wrote: summary, git SHA deadd6b, PRs #2–#10, the 1.0+1.5 branch decision.

The sidebar is the map: blog-updater, cdk-cost-killer, embabel-v1-learning, obsidian-mcp, and the rest of the lab. I do not keep one giant note. I keep one project folder and let the dated session files accumulate. When I open Cursor on that repo tomorrow, the first useful call is not “read the whole vault.” It is get_project_context for that slug — Project State plus the newest sessions and decisions.

The loop I run in Cursor

Cursor connected to an Obsidian vault through MCP stdio
Cursor (or Copilot) talks MCP stdio. The vault is the source of truth.

  1. Before substantial workget_project_context. Continuing a feature, debugging a known area, or answering “why is it like this?” Empty sections if the project is new. Never the entire vault.
  2. After meaningful workcapture_work_session with a short summary, changes, decisions, and next steps. Pass repository_path so the git snapshot lands in the same note.
  3. When a choice should stickrecord_decision. Example from that screenshot: keep Embabel 1.0 and 1.5 on one main; study from the cheat sheet, not a second cookbook.
  4. When overall status movesupdate_project_state. Concise. Current tense.

Lookup is local: search_memory over that project’s files, read_note for one vault-relative path, append_daily_note for a line that belongs on today’s Daily/YYYY-MM-DD.md instead of a project folder.

Retrieve context, implement, capture session, update project state
Retrieve → implement → capture → update state. Skip the capture when the change was a typo.

What is not persisted

The Cursor rule that ships with the installer is the product as much as the tools. Typo fixes, formatting-only edits, and one-line mechanical changes are not memory. If every keystroke becomes a session section, I will stop reading the vault — and so will the next agent.

Secrets never land in the vault on purpose. Values that look like keys, tokens, JWTs, or password= assignments are replaced with [redacted-secret] before write. I also never persist .env contents, database credentials, or customer data. Summarize the incident; do not paste the token.

Paths are confined to OBSIDIAN_VAULT_PATH. Absolute note paths, ../ traversal, and symlink escapes are rejected. This is not a general filesystem API. Writes are atomic.

Wire it once

export OBSIDIAN_VAULT_PATH="$HOME/Documents/ObsidianVault"
uv sync

./scripts/install-project.sh \
  --project /path/to/your-app \
  --vault "$OBSIDIAN_VAULT_PATH"

The installer merges .cursor/mcp.json (it does not wipe unrelated servers) and drops the Cursor rule plus Copilot instructions so both assistants use the same habits. Point the env var at a real vault directory. The server does not auto-load .env files.

I already have a product-shaped write-up of the seven tools. This post is the part I needed after the first week of using it: the vault is how I keep Cursor sessions and ongoing work in one place I can see. Open Obsidian when you want the graph. Leave it closed when you just need the assistant to remember the last SHA.

Source: github.com/jmjava/obsidian-mcp

Friday, August 28, 2026

The DIF test engine — prove the three layers without collapsing them

A plan that cannot fail in an interesting way is just more markdown. The last post said DIF, the orchestrator, and Embabel answer different questions. This post is the test engine that keeps them from collapsing into one runtime.

Follow-up to Three layers, one day. Source: the working test flow in jmjava/embabel-dif and the integration ladder in docs/ORCH_INTEGRATION_ROADMAP.md.

What “test engine” means here

Not a new product. Not a second daily driver. A stacked set of checks where each rung is allowed to fail before we spend complexity on the next:

./mvnw test                 # unit + CLI + FoldContractTest
                            # EmbabelLivePlatformTest skipped unless DIF_LIVE_EMBABEL=1
./scripts/dif-orch-smoke.sh # FEAT-001 ready + T03; FEAT-099 exit 1
./scripts/dif-orch-day.sh   # fold twice / architect / review / plan --projection
                            # skip when CLI or snapshots missing
./scripts/dif-live-e2e.sh   # orch Guide+Neo4j + JSONL quote + live GOAP

Default CI is the first two boxes. Live Guide and Embabel are opt-in. They reuse the orchestrator’s existing tests/test-guide-stack-live.sh. They do not put Embabel or Guide inside sdlc.sh next.

The engine’s job. Prove the three systems can talk. Prove a missing DIF checkout is skip, not a broken day. Prove a contradictory canvas cannot earn Ready For Coding. Never start a JVM to run next.

Rung A — five named checks, not prose

FoldContractTest is step 1 of the fold iteration plan. The five success criteria are tests:

Check What fails if we are wrong
Same accepted canvas → same model Nothing downstream is trustworthy
Review fails without “looks correct” login-auth-broken still prints RESULT: PASS
Syntax variance does not flip invariants A DTO rename (FEAT-070) changes what must stay true
Open T## is a MissingObligation T03 on FEAT-001 disappears into a checklist
Requirement vs non-goal blocks Ready For Coding FEAT-099 pagination clash still looks green

Harvested canvases under examples/canvases/ are the corpus. The fold learns from real REASONS files, not imagined IR. Adding a backend must not change the CLI or the canvas schema.

Rungs C–D — a script can trust the gate

dif-fold.sh writes a projection and a stable .gate.json. Smoke does not parse stdout for meaning. It reads JSON and exit codes:

{
  "workId": "FEAT-001-order-status-api",
  "readyForImplementation": true,
  "blockingConflicts": [],
  "missingObligations": ["T03"]
}

dif-orch-smoke.sh folds FEAT-001 (exit 0, ready, T03 missing) and FEAT-099 (exit 1, blocking pagination clash). If a sibling orch checkout is present, it folds the live examples/spring-boot-order-api canvas too. Then it hits the silent attach:

DIF_DISABLED=1 check-canvas.sh …   →  dif=skipped   (exit 0)
check-canvas.sh FEAT-001           →  dif=ready     (exit 0)
check-canvas.sh FEAT-099           →  dif=blocked   (exit 1)

One line. Agents do not get a fold dump. Missing DIF is skip, not a new ritual. That is the same opt-in shape as Guide.

Rung G — a scripted day, no Embabel

dif-orch-day.sh is the cheap “full day.” It does not start Embabel, Guide MCP, or replace sdlc.sh next.

  1. Fold the same FEAT-001 canvas twice. The two .gate.json files must cmp equal.
  2. Architect FEAT-001 → dif=ready, T03 still a missing obligation.
  3. Architect FEAT-099 → exit 1, dif=blocked.
  4. Review the orch order-status snapshots: dropping auth fails. A DTO rename still passes.
  5. plan --projection builds a VerificationPlan from the folded model. No markdown re-parse.
  6. Guide JSONL is an optional quote (Decision / Pitfall), not a gate.
  7. Missing CLI or missing snapshots → dif=skipped. Present snapshots that drop a safeguard → dif=blocked.

Review uses examples/snapshots/order-status-*.json and the canvas safeguard paths — not the old login fixtures. Syntax-ok vs auth-broken is the whole point: a rename is legal; a dropped safeguard is not.

Rungs H–J — live, still not inside next

dif-live-e2e.sh is the three-way path that already passed here. First it asserts sdlc-engine is not a JVM — help must not mention Spring or Embabel. Then it reuses the orch Guide+Neo4j harness, runs the scripted day, quotes DIF JSONL through GuideClient under a unique Work ID (FEAT-DIF-LIVE-… so it does not collide with orch’s already-projected FEAT-001), and boots the Embabel Spring platform.

Live Embabel (EmbabelLivePlatformTest, DIF_LIVE_EMBABEL=1) runs the fixture GOAP path:

UserInput
  → captureRequest → interpretIntent → foldIntent
  → analyzeRepository → planVerification
  → VerificationPlan (readyForImplementation, missing rotation IT)

The refresh-token wording uses FixtureIntentInterpreter — no LLM. A second test plans an already-folded orch canvas without re-parsing markdown. Conflicts stay on the VerificationPlan (readyForImplementation=false). They are not a GOAP precondition Embabel 1.5 cannot treat as an action post.

Orch CI does not need Maven. It uses tests/fixtures/dif-fold-stub.sh so detect-and-skip / fail-closed can be proven with a fake CLI: skipped, ready, or blocked.

What would fail the engine

The ladder is the falsification list from the last post, turned into commands:

  • Two folds of the same canvas disagree → day step 1 fails.
  • A requirement vs non-goal pair still looks ready → smoke / architect on FEAT-099 fails.
  • An open T03 does not show up → gate assertion fails.
  • A DTO rename flips an invariant → SyntaxVarianceTest / review syntax-ok fails.
  • Review cannot fail a dropped safeguard without “looks correct” → auth-broken still passes.
  • sdlc-engine --help mentions Embabel → live E2E step 0 fails.
  • Missing DIF breaks the orch day → skip tests fail.

If people stop reading the canvas because they treat the JSON as source of truth, we failed even if every script is green. The projection stays regenerable and disposable.

Source: github.com/jmjava/embabel-dif
Previous: Three layers, one day — DIF, the orchestrator, and Embabel
Related: sdlc-spdd-orchestrator · Embabel

Thursday, August 27, 2026

Three layers, one day — DIF, the orchestrator, and Embabel

Reliable AI engineering does not require every component to be deterministic. It requires determinism at the boundaries where repeatability, traceability, and correctness matter. That is the sentence embabel-dif is testing — not a Merly reimplementation, and not a second daily driver.

Use stochastic reasoning to discover knowledge. Use deterministic representations to operationalize it once it is understood.

The hole the runbook cannot close

Coding agents are good at reading a repository and sounding like they understand it. The understanding is usually implicit and disposable:

prompt + files + luck  →  a one-off theory of the system  →  a patch

The next session starts from zero. It may decide that sessionToken was incidental, that Google login can move, or that an existing test is optional. Nothing in the process remembers which of those beliefs were load-bearing.

sdlc-spdd-orchestrator already attacks the process half: one Work ID, one REASONS Canvas, one phase at a time. Assistants are not allowed to invent a parallel workflow. That is necessary and not sufficient. The canvas is still prose. Architect, review, and sync still ask an LLM to compare the canvas to a diff. Comparison is where implicit intent creeps back in.

Process gates ask “do the prerequisite files exist?” They do not ask “did this canvas contradict itself?” or “did this diff drop a safeguard?”

The remaining hole is checkability. You can follow the runbook perfectly and still ship a contradictory canvas, mark Ready For Coding in prose, or pass review because the change “looks right.”

Three questions, three systems

Planning / requirements     why are we doing this?
REASONS Canvas              what must ship (human contract)
DIF SemanticModel           what must remain true (machine contract)
Embabel GOAP                what action to take on typed facts (optional)
DICE / Guide graph          what did we learn before (retrieval)
SDLC phases                 who is allowed to act
Layer Owns this question Must not own
Orchestrator Who acts when? One Work ID, one canvas, one phase. Folding facts. Starting a JVM. Being a planner.
DIF What must stay true? Same accepted canvas → same model. Conflicts fail closed. Daily orientation. Picking the Work ID. Replacing the canvas.
Embabel What action to take on already folded facts (optional JVM path). The fold itself. sdlc.sh next. The human contract.

Git stores what changed. A DIF-style layer stores why it had to, and what must still be true. Embabel, when present, decides what to do next. The orchestrator decides who is allowed to act.

They stay three repos on purpose. Merging Embabel or DIF into the orchestrator would fight its design: it is an installable operating model, not a compiled agent runtime. The contract between them is a file:

spdd/canvas/<WORK-ID>.md            human source of truth
        │
        ▼  fold (deterministic after accept)
.dif/projections/<WORK-ID>.json     machine projection (disposable)
.dif/projections/<WORK-ID>.gate.json
        │
        └─ orch may read the exit code
           it does not start the JVM to run next

A canvas is already a candidate intent. We do not need a new human artifact. We need a projection.

DICE is not DIF

The orchestrator already has Guide DICE as an optional working store. The acronyms smash together. The jobs do not.

DICE  = retrieve what we already believe
        (lessons, decisions, pitfalls, area subgraphs)

DIF   = freeze what must remain true, then verify it
        (intents, invariants, conflicts, obligations)

DICE answers “what did previous work in this area learn?” DIF answers “may this change proceed, and did it preserve the contract?” Both can project from the same committed files. Neither replaces the canvas or the lessons ledger. The ledger stays the system of record; SQLite, Guide, and .dif/projections/ are regenerable.

What landed today

Yesterday’s prototype proved a typed fold on a refresh-token fixture. Today the fold attaches to real REASONS canvases and fails closed in a way a script can trust.

./mvnw test
./scripts/dif-orch-smoke.sh
./scripts/dif-fold.sh --canvas examples/canvases/FEAT-001-order-status-api.md
./scripts/dif-fold.sh architect --projection .dif/projections/FEAT-099-pagination-conflict.json
./scripts/dif-fold.sh review --before examples/snapshots/login-before.json \
                            --after examples/snapshots/login-auth-broken.json

dif-fold does not start Embabel. fold writes a projection and a stable .gate.json (readyForImplementation, blockingConflicts, missingObligations) that a script can read without parsing stdout. architect and review fail closed: exit 1 means not Ready For Coding, or invariants were not preserved.

After a fold, “ready” is allowed to mean this:

  • A mutually exclusive pair (“must paginate” vs “non-goal: pagination”) blocks Ready For Coding. The next command is clarification, not code. That is FEAT-099.
  • An open operation (T03) shows up as a MissingObligation, not a forgotten checklist box.
  • Two folds of the same accepted canvas produce the same model.
  • Syntax may change (DTO names, test style). Preservation of auth and unrelated endpoints must not. That is FEAT-070.
  • Review can fail a required safeguard without asking an LLM whether the change looks correct.

The ten fold-iteration steps from the steal list are implemented: fold contract tests, harvested canvases, heading classification, quoted conflicts, open-T## obligations, syntax-out-of-invariants, an optional Alloy sketch, architect/review attach, and a plan path that builds a VerificationPlan without making Guide required.

Take the idea only as far as it makes sense

The knowledge that actually hurts is not “which slash command is next.” The orchestrator already answers that. The tax is shipping a contradictory canvas or a dropped safeguard while the runbook stays green.

The filter for every attach:

Does this make the existing orchestrator commands harder to get wrong, without adding a new ritual?
Do Do not
Keep claim → next → architect → one T## → review as the only user surface Add dif-fold.sh next as a second daily driver
When DIF is installed, architect cannot earn Ready For Coding on a requirement vs non-goal clash Teach users fold / projection / .gate.json as a parallel workflow
When DIF is missing, the day is unchanged Require Embabel, Java, or OpenAI to run next
Review can fail a dropped safeguard without “looks correct” Replace sdlc.sh gate process checks with the fold

A new orchestrator user who never heard of DIF should have a better day if it is installed, and the same day if it is not. Silent fail-closed on existing architect / code is DIF doing DIF’s job: the readiness string becomes earned. The runbook stays the orchestrator’s. Embabel stays later and optional. Wiring it into next would be the other collapse.

Path, and what would falsify it

1. DIF      canvas → SemanticModel CLI      no Embabel          (working)
2. Orch     architect / code attach         if CLI present      (silent, opt-in)
3. DIF      Embabel GOAP for JVM targets    orch still picks Work ID / T##
4. Optional project invariants into Guide   shared vocabulary, still not required

Step 1 first: if the same canvas does not fold the same way twice, nothing downstream is trustworthy. Step 2 next: attaching an exit code is cheaper than inventing a new phase. Embabel later. Guide last — retrieval already works.

The idea is wrong if two folds disagree, if review still cannot fail a safeguard without “looks correct,” if a DTO rename flips a required invariant, if sdlc.sh next starts a JVM, or if developers need a second next to have a correct day. The projection must stay regenerable. If people stop reading the canvas, we failed even if the JSON is pretty.

Source: github.com/jmjava/embabel-dif
Publication plan: BLOG_DIF_ORCH_EMBABEL.md
Related: sdlc-spdd-orchestrator · Embabel · embabel-v1-learning

Sunday, August 23, 2026

Unreal Playground — Python designs a hole, Unreal and Blender build it

A golf hole should be a typed object before it is a mesh. courseforge/unreal-playground is a Python-first learning environment for AI-assisted course design. Agents write a validated CourseDesign. Unreal evaluates it in a game-engine world. Blender realizes it as a portable hole.glb. Then Python scores, critiques, and revises.

Python owns orchestration, schemas, scoring, storage, and the learning loop. Unreal and Blender are interchangeable backends — including a pure-Python fake so the whole loop runs without Docker or an editor.

The loop

prompt
  │
  ▼
DesignerAgent ──► CourseDesign JSON
  │                 (tee, green, fairway spline, hazards)
  ├──────────────► Unreal  evaluate  ──► metrics + camera PNGs
  └──────────────► Blender realize   ──► hole.glb + preview
  │
  ▼
PlayabilityAgent ──► score / critique ──► revise ──► next iteration

They are not two copies of the same renderer. Unreal answers “is this hole measurable as built?” Blender answers “what does this hole look like as geometry?” Both consume the same design object.

# Offline — no Unreal, no Docker
python python_env/orchestration/run_iteration.py \
  --prompt "short risk-reward par 4 in a Pine Barrens style" --fake

python python_env/orchestration/run_experiment.py \
  --prompt "links par 5 for a scratch golfer" --iterations 3 --fake

Each run lands under python_env/datasets/generated/<job_id>/: prompt, design JSON, job JSON, metrics, score, critique, screenshots, logs.

Where the engines actually are

Phase 1 is done. Pydantic schemas, deterministic designer/routing/critic agents, fake Unreal, run_iteration / run_experiment / run_batch, and dataset archival. The MVP hole is a short Pine Barrens par 4 you can generate on a laptop.

Phase 2 wired the CourseForge worker standard. A persistent unreal-worker runs the versioned unreal-golf-build job package. Backends swap without changing the agent graph: FakeUnrealRunner, WorkerUnrealRunner, one-shot Docker, or a host LocalUnrealRunner gated by COURSEFORGE_REAL_UNREAL=1.

Dual-engine graphics (FEAT-003) is the recent ship. Host Blender 4.x exports a real hole.glb plus Cycles preview. Host Unreal now goes past placeholder boxes:

  • fairway SplineComponent, rough and tree-line proxies
  • heightmap .r16 + real ALandscape import through a thin GolfCourse C++ bridge
  • PCG graphs + volume ISM fill when the commandlet cannot tick a StaticMeshSpawner
  • InstancedFoliage tree line from a saved foliage type

End-to-end Embabel iterations have run with both host binaries in the same job (Landscape + PCG on Unreal, glb/preview on Blender). Kind blender-worker already runs the same package over HTTP. Full in-pod Unreal golf jobs are still the open edge — the 35–49 GB worker image is a local/Kind concern, not CI.

Embabel on top of the same contracts

A Python mirror of Embabel’s nested-agent (Matryoshka) pattern lives in python_env/embabel_explore/. CourseDesignAgent nests design → evaluate → realize_blender → critique. Placement (fake / host binary / Kind worker) swaps underneath without rewriting the graph.

The gated designer studio (golf-embabel-web on port 8765) is chat + before/after, validation gates, and inline revise/redo. The same GUI can deploy into the CourseForge Kind suite. Cookbook recipes — type chaining, conditions, stuck recovery, RepeatUntil, thinking, streaming — are mapped onto this golf loop, not a second travel-agent demo.

Process sits on SDLC-SPDD canvases (FEAT-001 MVP hole, FEAT-002 worker integration, FEAT-003 dual-engine). The interesting remaining work is richer artist-authored PCG assets and a clean Kind Unreal worker run — not another schema rewrite.

Source: github.com/courseforge/unreal-playground
Related: embabel-v1-learning · SDLC-SPDD

CDK Cost Killer — tear down the stacks that keep billing

Forgotten CDK labs keep billing after you stop caring about them. cdk-cost-killer is a small CDK app that finds those stacks and tears them down — nightly for hygiene, and hourly only while you are already over budget.

It does not create a budget. It reads the account budget named Monthly budget (default limit $30; the live AWS Budgets number wins). The whole design is one Lambda, two schedules, and a few hard skip rules.

The loop

EventBridge Scheduler
   │
   ├─ 9:00 PM America/New_York  → action=kill, reason=nightly
   └─ every hour                → action=kill, reason=hourly
                                      │
                                      ▼
                              cdk-cost-killer Lambda
                                      │
                    ┌─────────────────┴─────────────────┐
                    │ read Monthly budget (actual vs cap)│
                    └─────────────────┬─────────────────┘
                                      │
              hourly + under budget? ─┤── yes → return (no-op)
                                      │
                                      ▼
                         scan enabled regions
                         group nested stacks by root
                         stop EC2 first, then DeleteStack

A third path is optional: subscribe the stack output BudgetAlertTopicArn to the budget’s 100% actual alert. SNS then invokes the same Lambda with reason=budget-notification instead of waiting for the next hourly check.

What is expensive enough to kill

A stack family is torn down only when it holds a resource that keeps costing money while it exists: EC2 instances, Elastic IPs, NAT gateways, VPC endpoints, load balancers, Auto Scaling groups, RDS, Redshift, ElastiCache, OpenSearch, EKS, ECS services.

Default scope is CDK only (AWS::CDK::Metadata or a CDK description). Set targetScope to all-cfn if you want any CloudFormation stack with those resources.

Always skipped:

  • this CdkCostKiller stack
  • CDKToolkit bootstrap stacks
  • anything tagged CostKillerProtect=true

EC2 in a doomed family is stopped first so compute charges drop while CloudFormation delete finishes. Elastic IPs and load balancers only stop costing money after the stack is gone.

Safety switches

Knob Default Why it exists
dryRun true Log would-stop / would-delete only. Arm after CloudWatch review.
enabled true Redeploy false to DISABLE both schedules and make the Lambda a no-op.
protectTagKey CostKillerProtect Tag keepers. Nested stacks inherit the root family’s decision.
regions all enabled Optional allowlist, e.g. us-east-1,us-east-2.
Hourly is not a second nightly. The hourly pass reads the live budget and bails if spend is still under the cap. After the billing period resets, those invocations become no-ops again. Nightly still runs regardless — that is the “I forgot to destroy the lab” cleanup.
npx cdk deploy                    # dry-run on by default
npx cdk deploy -c dryRun=false    # arm after reviewing logs
npx cdk deploy -c enabled=false   # off switch, stack stays

Tags.of(stack).add("CostKillerProtect", "true");

Source: github.com/jmjava/cdk-cost-killer

Embabel V1 Learning — one branch, two versions, filmed cheat sheet

Memorize the rules, then debug the agents. embabel-v1-learning is a study repo for the Embabel Agent Framework: Java and Kotlin side by side, guided unit tests, and a cheat sheet extracted from the official User Guide and Cookbook — not a second copy of the cookbook travel recipes.

The important constraint is one branch. Default Maven pins Embabel 1.0 (lessons 01–15). -Pembabel-15 compiles the 1.5 extras (thinking traces, streaming objects, message lists, tool-call inspectors) without turning main into a 1.5-only fork.

./mvnw test                 # Embabel 1.0 — lessons 01–15
./mvnw test -Pembabel-15    # plus thinking / streaming extras

The mental model in 30 seconds

input → blackboard types
      → planner picks next @Action
      → action may call LLM / code / tools / subagents
      → return value posts new types → REPLAN
      → @AchievesGoal return type → DONE

GOAP does not run your methods top-to-bottom. Types are the wiring. After every action Embabel reassesses the world (OODA). Returning a new object, null, or flipping a condition changes the next step. That is the first thing the top 10 asks you to internalize.

What the curriculum now includes

The study path is built so you can spend 60–90 minutes and actually remember something:

  • Cheat sheet + printable PDF — extracted API and planner rules. Keep it next to the debugger.
  • Lessons 01–15 — injected Ai, annotation agents, DICE tools, HITL, conditions/bindings, subagents, RepeatUntil, planner choice, @State loops, guardrails, stuck recovery, AgentInvocation, Kotlin DSL.
  • 1.5 extras — action cost, createObjectIfPossible, thinking, streaming, fromMessages, tool-call inspectors. Same @Action style; extra PromptRunner surface.
  • Review circuit — checkbox path: orient → debug Write/Review → tools/conditions/guardrails → planners.
  • Templates + snippets — copy-paste Java/Kotlin skeletons under templates/; type emb- in the editor (emb-agent, emb-hitl, emb-subagent, emb-dsl…).
Unit tests assert structure, not model poetry. Guided tests use FakeOperationContext: prompt contents, temperature, tool attachment, condition predicates. Live LLM judgment stays in the optional shells.

New: a filmed cheat sheet

The latest work is a Memory OS palace for the cheat sheet itself — not a narrated walkthrough of cookbook recipes. Two floors, twelve loci, one bronze type-ingot creature walking guessable body slots. Each organ is absurd and is the concept.

Floor What you walk
1 — 1.0 mental model Types as wiring, replan/OODA, mix code + LLM, tools attach per call, named bindings, four planner hats
2 — PromptRunner + 1.5 createObject, soft-fail null, message envelopes, thinking traces, streaming JSON bricks, tool inspectors

Published lengths: full film about 12 minutes; each floor about 5.5–5.7. The player is on GitHub Pages. Overlay labels and Q/A are stamped by the engine — stills are not allowed to paint lettering — so a rebuild on a newer Memory OS contract stays honest.

How to study it: skim the cheat sheet, watch a floor, then debug the matching *GuidedTest. The film is the retrieval hook; the test is the proof you understood the rule.

Source: github.com/jmjava/embabel-v1-learning
Player: jmjava.github.io/embabel-v1-learning
Related: Embabel Guide · memory-os

Memory OS — method-of-loci study films from Markdown

A study script should become a walk you can replay. That is the bet behind memory-os: write a palace in Markdown, compile it to a spec, and build a narrated, AI-illustrated method-of-loci film — one locus, one concept, one memorable image.

It is built on the pipeline we already proved in docgen (Markdown → TTS → declarative specs → ffmpeg), with the visual layer swapped to the OpenAI Images API so every room gets an exaggerated mnemonic instead of a Manim box diagram.

How the engine fits together

Humans write spec.md. The engine owns the rest.

your-palace.md
      │  memoryos compile
      ▼
your-palace.palace.yaml     # canonical spec
      ├─ enrich   → ~30s explanation per locus (LLM, reviewable in YAML)
      ├─ images   → images/<floor>/NN-locus.png
      ├─ narrate  → audio/<floor>/NN-locus.mp3
      └─ render   → build/video/<palace>.mp4

Sync is the core guarantee. Rendering is audio-first: one continuous narration track, every cut recorded in build/timeline.json, clips cut to those markers. A locus stays on screen until its explanation (plus a recall pause) finishes — never less than video.min_locus_sec (default 30 seconds). Image changes cannot drift ahead of the voice.

Recent engine work that mattered in practice:

  • Preview clips now have sound. Early builds encoded build/clips/*.mp4 video-only (-an). The MP3s were fine; the previews were silent. Each clip now muxes its own narration so a single scene is watchable, while the assembled film still uses one continuous AAC track.
  • Enrich is reviewable. Thin study-script lines expand to ~30s explanations inside the YAML. You edit the prose, not a generated MP4.
  • LAN + Pages viewers. memoryos serve binds on the LAN for iPad/Safari; memoryos pages emits a static tree with chapter / section / scene navigation and viewed/unviewed state.
  • Cheap, incremental images. Default is gpt-image-1-mini at medium quality (~$0.015/locus). Existing assets are reused unless you pass --force.

Dogfood: Spring Authorization Server palace

The first full palace is the Spring Authorization Server lab — the same design documented in examples/spring-auth-server/docs/c4-players.md. Vue and mobile are public clients. They talk only to Spring Authorization Server with PKCE. They never call Google or Apple token endpoints.

Token A — Google or Apple id_token. SAS is the IdP’s client.
Token B — Spring Auth id_token. The app is SAS’s client.
AT / RT — Spring Auth access and refresh tokens. These are what resource servers accept.

That three-token split is the whole point of filming it. Floor 1 already had stove = ID token (identity for the client, not an API credential) and refrigerator = access token (presented to APIs). The newer wings add the federation story: Apple/Google mint Token A for the auth server; the auth server mints Token B + AT/RT for the apps. You do not send Google’s id_token to your APIs.

The palace is specified as seven floors:

  1. OAuth / OIDC protocol — what happens
  2. Spring Authorization Server internals — which components make it happen
  3. Authorization code + PKCE
  4. Token and session lifecycle
  5. Debugging — where the request broke
  6. System flow — Vue, mobile, Cortex, Redis, ThreadLocal
  7. Config paths — the long AuthorizationServerConfig / SecurityConfig methods

Each floor also has a federation wing (Sign in with Google / Apple). Loci stay incremental: existing images and audio are reused unless you pass --force.

memoryos compile examples/spring-auth-server/spec.md \
  --id spring-auth-server \
  -o examples/spring-auth-server/spring-auth-server.palace.yaml

memoryos build examples/spring-auth-server/spring-auth-server.palace.yaml --floor floor-1
memoryos concat … --floors floor-1,floor-2,floor-3
memoryos serve … --concat

What is next

The engine is already a consumer library, not a one-off renderer. embabel-v1-learning pins it and publishes cheat-sheet films to GitHub Pages — two floors, twelve loci, about twelve minutes. Next work on this repo is more palace coverage (federation / refresh / Cortex session-token cache) and keeping the CLI contract stable so study repos can rebuild without vendoring the engine.

Source: github.com/jmjava/memory-os
Related: docgen · embabel-v1-learning · google-oauth-poc

Obsidian MCP — engineering memory that stays Markdown

Assistant memory should be files you can open. obsidian-mcp (0.1.0) is a local MCP server that gives Cursor and GitHub Copilot persistent engineering memory — stored as ordinary Markdown in an Obsidian vault. Obsidian does not need to be running. There is no community plugin and no hosted memory API.

The vault is the source of truth. Notes stay readable in Obsidian, git, or any text editor. The same stdio process works for both Cursor (.cursor/mcp.json) and Copilot / VS Code (.vscode/mcp.json).

The loop

Cursor / Copilot
        │  MCP stdio
        ▼
obsidian-dev-memory
        │
        ▼
Obsidian Markdown vault
  AI Memory/Projects/<slug>/
    Project State.md
    Sessions/YYYY-MM-DD.md
    Decisions/YYYY-MM-DD-<slug>.md
  1. Before substantial work, the assistant calls get_project_context.
  2. After a real implementation, it calls capture_work_session.
  3. When an architecture choice lands, it calls record_decision.
  4. When overall status changes, it calls update_project_state.

Typo fixes and one-line mechanical edits are not memory. The Cursor rule and Copilot instructions say so explicitly, so the vault does not fill with noise.

The seven tools

Tool What it does
get_project_context Project State plus newest sessions and decisions. Empty sections if the project is new — never the whole vault.
capture_work_session Append a timestamped section to today’s session note. Optional Git snapshot: repo, branch, short SHA, dirty flag, short file list. No full diffs.
record_decision Write YYYY-MM-DD-<slug>.md. Collision adds -2, -3 — never overwrite.
update_project_state Replace the concise current-state note. Not a session log.
search_memory Local filename and text search over that project’s memory.
read_note One vault-relative Markdown file.
append_daily_note Append to Daily/YYYY-MM-DD.md. Never overwrite existing contents.

Safety is the product

This is not a general filesystem API. Every note path must resolve inside OBSIDIAN_VAULT_PATH. Absolute paths, ../ traversal, and detectable symlink escapes are rejected. Writes are atomic (tempfile + os.replace).

Secrets never land in the vault on purpose. Values that look like keys, tokens, JWTs, private keys, or password= assignments are replaced with [redacted-secret] before write. The assistant instructions also forbid persisting .env contents, database credentials, and customer data.

Wire it into a project

export OBSIDIAN_VAULT_PATH="$HOME/Documents/ObsidianVault"
uv sync
uv run python -m obsidian_dev_memory

./scripts/install-project.sh \
  --project /path/to/your-app \
  --vault "$OBSIDIAN_VAULT_PATH"

The installer merges MCP JSON (it does not wipe unrelated servers) and drops the Cursor rule plus Copilot instructions so both assistants use the same memory habits. Point OBSIDIAN_VAULT_PATH at a real vault directory; the server does not auto-load .env files.

0.1.0 is the first cut that is worth installing: seven tools, vault confinement, Git context on sessions, and an installer that does not destroy existing MCP config. Next work is whatever the vault teaches us we forgot to remember.

Source: github.com/jmjava/obsidian-mcp

Saturday, August 08, 2026

SDLC-SPDD storage v3 — one folder, one ledger, query don't bulk-read

Agent memory only works if humans can audit it and models don't have to read the whole repo every session. That was the gap in early SDLC-SPDD dogfood: progress mirrors beside the ledger, feature folders under agent-context/, and “just grep the lessons file” retrieval. Storage v3 — now on main in sdlc-spdd-orchestrator — collapses the layout, commits to one JSONL ledger, and treats everything else as query or projection.

One folder to install

Framework-owned paths live under a single home: <repo>/sdlc-spdd/. Requirements, canvases, harness skills, installed scripts, and committed memory sit together; runtime state stays gitignored under .sdlc/.

WBS: single-folder sdlc-spdd install layout
09-install-layout — what setup-agent-prompts.sh / upgrade-project.sh lay down.

Legacy sprawled installs (framework dirs at repo root) still resolve until you run sdlc-engine storage migrate --consolidate. New projects get the single-folder layout from day one.

Ledger-first memory

The committed system of record is one append-only file: spdd/memory/lessons.jsonl. Work claim/release events go to spdd/memory/registry.jsonl. Neither file is hand-edited — agents stage via sdlc.sh capture and promote via sdlc.sh accept at retro/sync.

Class diagram: LessonRecord, LessonsLedger, registry events, projections
05-storage-model — one ledger; sqlite and Guide are downstream.

Stage quietly, accept at the gate

Captures land in gitignored .sdlc/staged/lessons.jsonl during coding and review. Nothing hits the committed ledger until retro or sync runs accept — one batched promotion instead of capture noise in git history.

Sequence: capture to staged, accept promotes to committed ledger
06-stage-then-accept — the quiet capture / loud accept split.

Projections you can rebuild

Optional backends — local SQLite (.sdlc/index.sqlite) and Guide DICE (Neo4j via orch-guide) — are regenerable projections of the same ledger. One write path; sdlc-engine context parity checks they still match.

Sequence: write ledger once; sqlite and Guide derived; parity repair
08-projection-parity — if Guide is down, files still win.

Lifecycle with storage gates

The SPDD hybrid lifecycle (Initialize → Analysis → … → Retro → Sync) now has explicit storage checkpoints: analysis indexes land in staged memory; retro/sync accept promotes keepers before the next Work ID.

Activity: SPDD phases with stage and accept gates
04-lifecycle-flow — when memory moves from hot session to durable ledger.

What sits inside the home folder

Adapters talk to Cursor/Copilot/Claude; the Python engine owns workflow, persistence config, and optional Guide ops; shell scripts remain the supported install path for consumers.

C4 container diagram: adapters, engine, ledger, scripts inside sdlc-spdd
02-container — the moving parts after v3.

Mental model in one line: canvases and requirements are read directly; the lessons ledger is the committed record; Guide and SQLite are working stores you query on demand; captures stage quietly and accept at gates. Full spec: docs/storage-v3.md. All diagrams: docs/diagrams/.

What we shipped to get here

  • PR #141 — storage v3 on main: single-folder install, JSONL ledger + registry, staged captures, test-suite restructure (unit / integration / e2e), migration tooling.
  • Consumer matrix — live install scenarios green; two intentional skips where dogfood canvas is empty (documented, not failures).
  • PR #148 (in flight) — ADF template library + Vue3 ops console rebased onto v3 paths; closes stale PR #115.

Try it

  1. Read storage v3 and skim the PlantUML sources (./scripts/render-diagrams.sh regenerates SVG).
  2. On an existing install: sdlc-engine storage status then migrate when ready.
  3. Opt into Guide only when you want cross-work graph retrieval — file indexes remain the baseline.

— John · github.com/jmjava/sdlc-spdd-orchestrator