← Case studies

Engineering rigor

Chess trainer

I built a chess trainer that will not show you a move the engine won't back: if Stockfish rejects the line the language model just wrote, the lesson is dropped rather than displayed.

My role

Sole author. I wrote the spec, froze the contracts, and wrote the acceptance script for each of the twelve goals; coding agents implemented against those scripts inside a harness I designed, and I reviewed and gated every goal before it landed. Fifty-four commits, one author. No team, no client, no external contributor.

The decision

Every "AI chess coach" asks a language model what went wrong. You get fluent, confident, frequently false analysis, and a beginner cannot tell the difference — which is precisely who the tool is for.

Two alternatives lost. A heuristic rulebook (material counts, opening book, piece activity) is cheap and offline, but it cannot answer the one question that matters: was this move bad in this position. Spot-checking the model's output loses for a subtler reason — a guarantee you enforce sometimes is not a guarantee.

So the engine is the source of truth and the model is a translator. Stockfish decides what the mistake was and what the best move is; the model only renders that into prose. Every move the model names passes through a frozen validate_move(fen, move) before it can reach the screen, and a rejected move raises rather than degrading gracefully. The rule is enforced by a test, not by a comment — a principle in a doc erodes; a failing test does not.

Proof

236 automated tests. The engine seam is a protocol both implementations satisfy: the real UCI subprocess wrapper, and a deterministic test double that raises on any position a test didn't explicitly stub rather than returning a plausible default. Golden centipawn values were seeded from a genuine Stockfish run and frozen, and the labelled blunder ground truth is pinned in the test rather than re-derived from the code under test. Exactly one test spawns real Stockfish.

CI runs against a real Postgres and an installed Stockfish with the skip-guards flipped to fail, so a test that would skip for a missing backend fails the build instead. Each of the twelve goals has its own acceptance script, and a ratchet re-runs all of them. Run against my own archive: 213 games, 11,582 plies.

Code is private; walkthrough available on request.

What it shows I build things that can be checked, not just demoed.

Honest limitations

It has one user: me — by design, not omission. No accounts, no multi-user; drill progress lives in browser storage. The dashboard was deployed behind basic auth because it publishes my own mistakes; I have since parked the project and it is not running now, so there is no public demo to click. 236 is the collected count: without a live Postgres and a Stockfish binary, 219 run and 17 skip — the suite only goes fully green in CI. And it exercises engine logic through the fake; the real engine is in the product, not in the inner loop of the tests. Commercially it is a non-starter — the incumbent is owned by Chess.com and ships the same loop. I built it because the problem was interesting.

Stack

Python 3.12, three runtime dependencies. python-chess for board and PGN replay only, never as an evaluator. Stockfish over a hand-rolled UCI subprocess with guaranteed reaping. Postgres via psycopg, in the one module allowed to know SQL exists. Static HTML/JS front end, no framework. ruff, mypy strict, pytest, GitHub Actions.