Showing posts with label tooling. Show all posts
Showing posts with label tooling. Show all posts

Sunday, September 27, 2026

Five additions that improve maintainability and code quality

The goal was to improve maintainability and code quality across the repositories I ship. The two are not the same thing, and both matter. Maintainability is the cost of the next change: how long it takes an engineer to open the code, make a change, and move on. Code quality is whether that change did what they meant and nothing else moved. Low quality makes every change a risk. Low maintainability makes every change slow. In September 2026 I made five additions to every repository to improve both. This post is about what was added, what each addition does, and how the work is using them.

Updated 11 October 2026. The first version of this post, earlier the same day, counted nine repositories with all five checks on the default branch. That count is now 16 of 17.

Why a score for the whole repository fails

The common approach is one number for the whole code base. Every file is graded, and the build goes red when the total is over a line. It sounds like discipline. In practice the number is driven by old files nobody is changing. To move it, an engineer opens a file the product does not need touched, rewrites it, and writes tests for the rewrite. The suite grows. The number moves. Maintainability did not improve, because the next real change was never in that file. Quality did not improve either, because a rewrite of working code is a new place for a defect. The team is working for the number.

The additions below take a different route to both. Each one grades the change in front of it. Code the change did not touch is left alone. Problems that already exist stay written down, and nobody is allowed to delete the list to make the build pass.

The five additions

Each addition is one check in the build, and each check is one question. A manager can ask it of any change and get a yes or a no.

1. Did this change add a new problem on the lines it touched?
Only the lines this change wrote are graded. A small edit in an old, tangled file passes if the edit itself is clean. A new problem on a changed line fails.
Maintainability: the code stops getting worse where people are working. Every change that lands is at least as clean as what it replaced, so the next change in that file is no harder than this one.
Quality: the problems this check catches are the ones that hide defects: a branch nobody can follow, a function doing four jobs, a value used before it is checked. Refusing them on the changed line keeps them out of the product.

2. Is this file both tangled and the one people keep editing?
A file is called out only when both are true. Hard to change, and changed often. Difficulty alone is not a project. A tangled file nobody opens costs nothing until somebody opens it.
Maintainability: cleanup goes to the file that is slowing down every edit. That is where a planned cleanup buys back the most time, and the rest of the debt is left where it is.
Quality: a file that is both tangled and edited often is where defects come from. Every edit to it is a chance to break something nobody can see. Cleaning that file, and only that file, removes the most defect risk for the least work.

3. Did this change break one rule the repository is not allowed to break?
Each repository names one rule that matters more than any style score, and tests it in the build. A post stays a draft unless publishing is turned on. A log must not contain a password. A fork must not grow a path back to the project it came from.
Maintainability: the rule holds without anyone having to remember it. A new engineer, or a busy one, cannot break it by accident, and nobody has to re-learn it by cleaning up after a mistake.
Quality: these are the defects that matter most and are found latest. A post that went live, a password in a log, a change sent to the wrong project. The check turns each one from an incident into a red build.

4. Did someone edit the written list of old problems so the build would pass?
Existing problems are recorded. A new one fails the build. The list may shrink when a problem is actually fixed. It may not shrink because the build was red.
Maintainability: the debt stays visible and readable, so decisions about it are made on facts rather than on a number someone adjusted.
Quality: a green build means the change was clean. It does not mean the list was trimmed. Without this check the other four can be switched off quietly, and the quality signal from the build is gone.

5. Do the tests for one chosen piece of the product still catch a mistake they used to catch?
The build breaks that piece on purpose and expects the tests to notice. If they do not, the build fails. Each repository checks one piece, because doing this to everything on every change costs more than it returns. A check that tests nothing fails.
Maintainability: an engineer can change that piece and trust the tests to catch what they broke. A suite nobody trusts makes every change slower, because the engineer has to check by hand what the tests should have checked.
Quality: a large test suite is not a safe one. Tests that pass no matter what the code does give a green build and no protection. This check is the only one of the five that measures whether the tests can find a defect at all.

How the additions compound

None of these is dramatic on its own. Together, over months, they bring the cost of the next change down and the defect rate down with it.

On maintainability: the parts of the product that get the most attention are also the parts the checks run on most often, so they get steadily easier to change. Old debt does not leak into new work, because the first check refuses it at the line. The second turns cleanup from a reaction to a score into a decision about a named file with a known cost.

On quality: the fifth check keeps the suite honest, so a green build carries information. The fourth keeps the record honest, so the trend is real. The third holds the rules that a busy team would otherwise re-learn by breaking them in production. Together they mean that when the build says a change is fine, it is fine, and when it says stop, there is a defect or a risk behind it.

What a manager should expect to see: fewer surprises in review, because the build already refused the change that made things worse; cleanup work that can be explained in one sentence, because a check named the file; and a build that is red for a reason someone can act on today, rather than for a number the team has learned to route around.

What this does not do: it does not pay down the existing debt on a schedule, and it does not promise the number goes to zero. The debt is written down and left alone until a change touches it or a hotspot earns a cleanup. That is on purpose. The alternative is a team working for the number.

What it costs

No new platform, no new licence, no new role. Each check uses the grading tool the language already has, running inside the build the repository already runs. The first four add minutes to a build. The fifth is the expensive one, which is why it is limited to one piece of the product per repository and not the whole thing.

There is a one-time cost when the checks arrive. The record of existing problems has to be written down once, and someone has to read the first hotspot list and decide which file, if any, earns a cleanup. After that, the cost is paid by the engineer whose change turned the build red, on that change, the same day. It does not become a backlog.

The cost that goes away is the one nobody was tracking: the hours spent on cleanup the product did not ask for, the tests written to cover that cleanup, and the review time spent on both.

How to tell it is working

A manager does not need to read the code to see whether this is holding. The signals are in the build and the review queue.

  • The record of old problems only shrinks when a fix lands. If it drops without a corresponding change, someone edited it, and the fourth check should have failed. If it never drops, the hotspot list is not being acted on.
  • Hotspot cleanups can be named. Each one points at a file that was both tangled and edited often. If cleanup work cannot be tied to a named file from the list, it is the old behaviour coming back.
  • Red builds are fixed inside the change that caused them. Not as a follow-on ticket, not as a cleanup epic. If red builds are turning into tickets, the checks are grading the wrong thing.
  • Review comments move from style to behaviour. The build has already said no to the change that made the code worse. Reviewers can spend their attention on whether the change does what it claims.
  • Time from open to merge does not grow as the code base grows. That is the cost of the next change, measured directly. It is the number this whole thing exists to hold flat.

If the record shrinks honestly, cleanup has a name, red builds are fixed in place, and merge time holds, the checks are doing their job. If the team starts routing around a check, that is the signal to look at the check, not the team.

How the work is using them

The checks land the same way the work does. A repository gets the one check it is still missing. That change stays only when its build is green. A red build is fixed inside the change that caused it. The written list of old problems is not edited so a build will pass.

As of the evening of 11 October 2026, 16 of 17 repositories run all five checks on the default branch. The nine from the first version of this post are still in that set: the documentation generator, the lifecycle orchestrator, the notes server, the chatbot, the learning project, the cost tool, the graph service, the delivery pipeline, and the installer. Seven more finished the same day: the model-setup project, the commerce platform, both guide forks, the assistant, this blog, and the indicator. On all sixteen, every change is graded on the lines it touched. A file is called out only when it is both tangled and edited often. One rule is tested in the build. Old problems stay written down. One piece of the product is broken on purpose, and the tests have to notice.

The rules that were still being added this afternoon are now in the build. On the commerce platform, calls into the commerce engine go through one client. On both guide forks, nothing grows a path back to the project they were forked from. On the assistant, a skill file has to carry the fields the catalog requires. On the indicator, two files dropped in the same second resolve to the later one. On this blog, a post stays a draft unless publishing is turned on, and a log must not contain a credential.

The one repository still short is the memory project. Its one-rule check is waiting, because the file it would touch is both tangled and edited often. That wait is the hotspot check doing the job it was added to do. The camera project already runs the hotspot check, the written list, and the test check. The language project already runs changed-line grading and the hotspot check.

Two products are off this list on purpose. The guide work stays on the fork. Changes are not sent back to the project that fork came from.

Where the checks run

The same five checks, in the form each language already supports, on every repository below. Sixteen of the seventeen on this program run all five on every change. The memory project is the one still short a check. The camera project and the language project are on the list and outside that 17.

Friday, July 17, 2026

Introducing docgen — narrated demos from Markdown to Manim

Long-form demos should explain how a system works. Narrated diagram videos age well when the script and the visuals are first-class artifacts you can regenerate in CI — not one-off recordings that rot with every UI tweak.

docgen (documentation-generator) is a reusable Python library and CLI for that job: Markdown narration, OpenAI TTS, Whisper-aligned timing, Manim scenes, ffmpeg composition, and validation you can run before you ship. Install it, point it at a docgen.yaml, and build demos from the shell — no IDE plugin required.

docgen as a reusable CLI library for narrated demo videos
Library, not app: pip-installable CLI + YAML + shell/CI.

What it is

docgen ships the video stack you need for scripted explainers:

  • TTS narration — Markdown scripts → MP3 via OpenAI (gpt-4o-mini-tts)
  • Whisper-style timestamps — word-level timing so visuals can wait on real speech
  • Manim animations — the primary visual surface for diagram-heavy segments
  • ffmpeg compose / concat — mux audio + video, stitch segments, freeze-tail guard
  • validate — A/V drift, freeze ratio, narration lint, Manim layout hints, pre-push checks
  • pages — static preview HTML for demo assets
  • wizard — optional local web UI to bootstrap narration from project docs

North-star constraints matter as much as features: stable CLI contracts, hybrid config (deterministic merges plus optional OpenAI where it helps), and a hard rule that generated assets come from the tool — not hand-edited “fixes” that paper over generator gaps.

docgen pipeline from narration through TTS, timestamps, Manim, and compose
Typical path: narration → TTS → timestamps → Manim → compose → validate.

cd your-project/docs/demos
docgen yaml-generate          # merge hints/defaults into docgen.yaml
docgen narration-generate …   # optional LLM narration from hints
docgen scene-spec-generate …  # declarative Manim YAML
docgen generate-all           # TTS → timestamps → Manim → compose → validate
docgen validate --pre-push

What we shipped recently

1. Declarative Manim: scene-spec-generate + scene-compile

Instead of hand-editing generated Manim classes, maintainers steer with hints and declarative *.scene.yaml specs. OpenAI can emit the YAML; the engine compiles it into _TimedScene classes inside marked regions of scenes.py.

Declarative Manim scene YAML compiling into animated diagram boxes
YAML in, timed Manim scenes out — with layout budgets and Whisper wait_word alignment.

The compiler is opinionated in useful ways: rows auto-paginate when they exceed the frame stack budget, oversized specs are rejected, and (when timing.json has Whisper words) each row’s first label can map to a wait_word index so boxes appear with the narration.

docgen scene-spec-generate --segment 01 --compile
docgen scene-compile animations/specs/01-overview.scene.yaml
docgen manim --scene YourGeneratedScene

2. Hints + yaml-generate as the maintainer surface

Demo bundles (typically docs/demos/) prefer hint files with YAML front matter over ad-hoc surgery on merged docgen.yaml. docgen yaml-generate merges segment lists, visual_map, and paths; narration-generate and scene-spec-generate read those hints. Generated narration, scenes, audio, and recordings stay tool-owned so Git review stays honest.

3. Handbook diagrams + Pages-friendly demos

The repo also ships a suite handbook under docs/suite/ (PlantUML sources with Graphviz/CI rendering) and Manim-oriented demo media for GitHub Pages — so architecture stories can ship as diagrams and narrated segments, not only as markdown.

Try it

  1. Install from source or git: see jmjava/documentation-generator
  2. docgen init a demos bundle (or adopt an existing docs/demos/)
  3. Author hints → yaml-generate → narration / scene specs → generate-all
  4. Run docgen validate --pre-push before you ship media

If you want demos that explain architecture with speech and diagrams — and you want a pipeline you can re-run instead of re-record — start with a Manim segment and let docgen own the rest.

— John · github.com/jmjava/documentation-generator

Introducing SDLC-SPDD — disciplined AI delivery, and what we shipped

AI coding assistants are fast. Delivery still needs a spine. Without one, sessions drift, Work IDs get lost, and “what should we do next?” becomes a new chat every morning.

SDLC-SPDD Orchestrator is a multi-assistant scaffold for disciplined AI-assisted delivery. It installs into your project and gives Cursor, Copilot, and Claude Code the same operating model: plan why the work matters, design what to build (and what not to), then run phases with clean handoffs.

SDLC-SPDD three parts: Planning, REASONS canvas, SDLC phases
Three parts that work together: Planning, SPDD (REASONS canvas), and SDLC lifecycle.

The idea in one minute

The framework is built from three parts:

Part Answers Artifacts
Planning Why the work matters ROADMAP.md, milestones, requirements, session notes
SPDD What to build (and what not to) REASONS canvas under spdd/canvas/
SDLC Who acts when and how sessions hand off phase commands, session briefs, agent-context/ memory

Commands come in two flavors: assistant slash commands in chat (/sdlc-spdd-plan, /sdlc-spdd-code, …) and a shell workflow CLI for day-to-day orientation (./scripts/sdlc.sh next). Install once from the orchestrator repo; then work inside your target project.

We develop the framework the same way we ask adopters to work: Work IDs, canvases, and milestones — dogfooding on the orchestrator itself. Progress follows Kent Beck’s ladder: make it work → make it right → make it fast.

What we shipped recently

The biggest recent theme is agent coordination — so humans and assistants stay on the same Work ID and the same next step.

Where am I workflow orientation for AI assistants
Orientation beats vibes: the same “what now?” answer in the shell and in chat.

1. SDLC pointer + workflow CLI

A persistent Work ID on your machine (.sdlc/pointer) and guarded wrappers that refuse commands aimed at the wrong chore. The workflow CLI tracks phase and gates, infers the next canvas operation, and captures sessions safely:

./scripts/sdlc.sh next
./scripts/sdlc.sh advance
./scripts/sdlc.sh shelf
./scripts/sdlc.sh resume

Shipped via #20 and #21 (tracking #19).

2. /sdlc-spdd-whereami (and friends)

In Cursor, Copilot, and Claude you can ask /sdlc-spdd-whereami and get the same orientation as sdlc.sh next. Workflow chat wrappers also cover claim, shelf, advance, next, and team — so you do not have to leave the assistant to stay in sync.

Team work-ID registry and claims synced through git
Team claims live in git so ownership is visible across machines.

3. Team Work ID registry

Shared claims in agent-context/work-registry.tsv sync through git. Teammates can see who owns which Work ID, with stale TTL and notes for branch / PR / Jira:

./scripts/sdlc.sh claim FEAT-005
./scripts/sdlc.sh team
./scripts/sdlc.sh release FEAT-005

Pointer and workflow state stay local; team claims are the shared layer.

4. Milestone 1 — make it right (and measure make it fast)

On the make-it-right track we completed the structural cleanup adopters feel every day:

  • Shared script library — less duplicated shell logic across install/workflow scripts
  • Command-spec generation — adapters stay aligned across Cursor / Copilot / Claude
  • Extension hook manifest — clearer places for project-specific overrides
  • Analysis Scope Lock — /sdlc-spdd-analysis locks IN / NOT scope before generation
  • Jira-compatible requirements — YAML frontmatter + validation for milestone docs
  • Milestone subdirectory layout — cleaner requirements/milestones/ structure
  • Session-brief archive — rotation so session folders do not grow forever

For make-it-fast measurement, we landed a prompt-optimization ledger and canvas readiness indicators — so coding gates on “Ready For Coding” instead of hope. Remaining make-it-fast spikes (Guide RAG context backend, local models) stay experimental until Guide MCP is ready.

Try it

  1. Clone jmjava/sdlc-spdd-orchestrator
  2. Install into a target project with ./scripts/setup-agent-prompts.sh (see the README adoption path)
  3. In chat: /sdlc-spdd-whereami — or in the shell: ./scripts/sdlc-spdd/sdlc.sh next
  4. Optional: watch the short narrated demos on GitHub Pages

If you are already deep in AI-assisted coding and want fewer lost threads between sessions, start with the first-day walkthrough in the repo docs — then claim a real Work ID and let the canvas carry the contract.

— John · github.com/jmjava/sdlc-spdd-orchestrator