Thursday, August 27, 2026

Three layers, one day — DIF, the orchestrator, and Embabel

Reliable AI engineering does not require every component to be deterministic. It requires determinism at the boundaries where repeatability, traceability, and correctness matter. That is the sentence embabel-dif is testing — not a Merly reimplementation, and not a second daily driver.

Use stochastic reasoning to discover knowledge. Use deterministic representations to operationalize it once it is understood.

The hole the runbook cannot close

Coding agents are good at reading a repository and sounding like they understand it. The understanding is usually implicit and disposable:

prompt + files + luck  →  a one-off theory of the system  →  a patch

The next session starts from zero. It may decide that sessionToken was incidental, that Google login can move, or that an existing test is optional. Nothing in the process remembers which of those beliefs were load-bearing.

sdlc-spdd-orchestrator already attacks the process half: one Work ID, one REASONS Canvas, one phase at a time. Assistants are not allowed to invent a parallel workflow. That is necessary and not sufficient. The canvas is still prose. Architect, review, and sync still ask an LLM to compare the canvas to a diff. Comparison is where implicit intent creeps back in.

Process gates ask “do the prerequisite files exist?” They do not ask “did this canvas contradict itself?” or “did this diff drop a safeguard?”

The remaining hole is checkability. You can follow the runbook perfectly and still ship a contradictory canvas, mark Ready For Coding in prose, or pass review because the change “looks right.”

Three questions, three systems

Planning / requirements     why are we doing this?
REASONS Canvas              what must ship (human contract)
DIF SemanticModel           what must remain true (machine contract)
Embabel GOAP                what action to take on typed facts (optional)
DICE / Guide graph          what did we learn before (retrieval)
SDLC phases                 who is allowed to act
Layer Owns this question Must not own
Orchestrator Who acts when? One Work ID, one canvas, one phase. Folding facts. Starting a JVM. Being a planner.
DIF What must stay true? Same accepted canvas → same model. Conflicts fail closed. Daily orientation. Picking the Work ID. Replacing the canvas.
Embabel What action to take on already folded facts (optional JVM path). The fold itself. sdlc.sh next. The human contract.

Git stores what changed. A DIF-style layer stores why it had to, and what must still be true. Embabel, when present, decides what to do next. The orchestrator decides who is allowed to act.

They stay three repos on purpose. Merging Embabel or DIF into the orchestrator would fight its design: it is an installable operating model, not a compiled agent runtime. The contract between them is a file:

spdd/canvas/<WORK-ID>.md            human source of truth
        │
        ▼  fold (deterministic after accept)
.dif/projections/<WORK-ID>.json     machine projection (disposable)
.dif/projections/<WORK-ID>.gate.json
        │
        └─ orch may read the exit code
           it does not start the JVM to run next

A canvas is already a candidate intent. We do not need a new human artifact. We need a projection.

DICE is not DIF

The orchestrator already has Guide DICE as an optional working store. The acronyms smash together. The jobs do not.

DICE  = retrieve what we already believe
        (lessons, decisions, pitfalls, area subgraphs)

DIF   = freeze what must remain true, then verify it
        (intents, invariants, conflicts, obligations)

DICE answers “what did previous work in this area learn?” DIF answers “may this change proceed, and did it preserve the contract?” Both can project from the same committed files. Neither replaces the canvas or the lessons ledger. The ledger stays the system of record; SQLite, Guide, and .dif/projections/ are regenerable.

What landed today

Yesterday’s prototype proved a typed fold on a refresh-token fixture. Today the fold attaches to real REASONS canvases and fails closed in a way a script can trust.

./mvnw test
./scripts/dif-orch-smoke.sh
./scripts/dif-fold.sh --canvas examples/canvases/FEAT-001-order-status-api.md
./scripts/dif-fold.sh architect --projection .dif/projections/FEAT-099-pagination-conflict.json
./scripts/dif-fold.sh review --before examples/snapshots/login-before.json \
                            --after examples/snapshots/login-auth-broken.json

dif-fold does not start Embabel. fold writes a projection and a stable .gate.json (readyForImplementation, blockingConflicts, missingObligations) that a script can read without parsing stdout. architect and review fail closed: exit 1 means not Ready For Coding, or invariants were not preserved.

After a fold, “ready” is allowed to mean this:

  • A mutually exclusive pair (“must paginate” vs “non-goal: pagination”) blocks Ready For Coding. The next command is clarification, not code. That is FEAT-099.
  • An open operation (T03) shows up as a MissingObligation, not a forgotten checklist box.
  • Two folds of the same accepted canvas produce the same model.
  • Syntax may change (DTO names, test style). Preservation of auth and unrelated endpoints must not. That is FEAT-070.
  • Review can fail a required safeguard without asking an LLM whether the change looks correct.

The ten fold-iteration steps from the steal list are implemented: fold contract tests, harvested canvases, heading classification, quoted conflicts, open-T## obligations, syntax-out-of-invariants, an optional Alloy sketch, architect/review attach, and a plan path that builds a VerificationPlan without making Guide required.

Take the idea only as far as it makes sense

The knowledge that actually hurts is not “which slash command is next.” The orchestrator already answers that. The tax is shipping a contradictory canvas or a dropped safeguard while the runbook stays green.

The filter for every attach:

Does this make the existing orchestrator commands harder to get wrong, without adding a new ritual?
Do Do not
Keep claim → next → architect → one T## → review as the only user surface Add dif-fold.sh next as a second daily driver
When DIF is installed, architect cannot earn Ready For Coding on a requirement vs non-goal clash Teach users fold / projection / .gate.json as a parallel workflow
When DIF is missing, the day is unchanged Require Embabel, Java, or OpenAI to run next
Review can fail a dropped safeguard without “looks correct” Replace sdlc.sh gate process checks with the fold

A new orchestrator user who never heard of DIF should have a better day if it is installed, and the same day if it is not. Silent fail-closed on existing architect / code is DIF doing DIF’s job: the readiness string becomes earned. The runbook stays the orchestrator’s. Embabel stays later and optional. Wiring it into next would be the other collapse.

Path, and what would falsify it

1. DIF      canvas → SemanticModel CLI      no Embabel          (working)
2. Orch     architect / code attach         if CLI present      (silent, opt-in)
3. DIF      Embabel GOAP for JVM targets    orch still picks Work ID / T##
4. Optional project invariants into Guide   shared vocabulary, still not required

Step 1 first: if the same canvas does not fold the same way twice, nothing downstream is trustworthy. Step 2 next: attaching an exit code is cheaper than inventing a new phase. Embabel later. Guide last — retrieval already works.

The idea is wrong if two folds disagree, if review still cannot fail a safeguard without “looks correct,” if a DTO rename flips a required invariant, if sdlc.sh next starts a JVM, or if developers need a second next to have a correct day. The projection must stay regenerable. If people stop reading the canvas, we failed even if the JSON is pretty.

Source: github.com/jmjava/embabel-dif
Publication plan: BLOG_DIF_ORCH_EMBABEL.md
Related: sdlc-spdd-orchestrator · Embabel · embabel-v1-learning

No comments: