Every public repository, read as a case file: the problem it solves, the approach taken, how it was validated, and what it taught. Shipped repos link to source; planned ones show the intended method, not invented results.
overkillACTIVE
AI EVALUATION
PROBLEM
A model grading its own design proves nothing, and a chain-reaction machine is easy to fake: a ball that would have fallen onto the switch anyway looks just like a working chain.
APPROACH
One TypeScript core runs the same checks in Node and in the browser: a JSON blueprint goes into Rapier 2D's deterministic build, and a part joins the chain only if the push changed its path, measured against a push-free twin of the machine stepped in lockstep. A success needs 5+ chained parts reaching a finale that stays untouched when nothing pushes. The site's machines were designed offline by Claude Opus 5.5 with the same tools and checks; live generation needs an API key and isn't running.
VALIDATION
230 unit tests · 6 end-to-end tests on the built site. The suite re-simulates all 19 stored attempts, and each machine re-runs in the visitor's browser with its trace hash shown next to the one recorded in Node. An independent review tightened the chain rule; no stored result changed. MIT.
FINDINGS
10 of the 20 eval chores are on the site, each working within 1–3 attempts, with all 9 failed revisions kept. The offline designer could read the source, engine geometry included, so this is not an eval of the live agent. Two machines are weak: 'let the dog out' chains 5 of its 11 parts, and 'turn off the light' is mostly dominoes.
LESSON
Let the simulator be the judge, and say exactly what an offline run can't prove.
AI-assisted builds drift from intent without a spine: no gates, no acceptance criteria, no evidence trail.
APPROACH
A stage-gated lifecycle driven by executable acceptance specs, with reusable templates and checklists per stage.
VALIDATION
Built with parallel AI agents, then put through an independent max-effort audit (graded B+); findings fixed and re-verified. markdownlint + link-check CI.
FINDINGS
Auto-reviewers will confidently 'fix' things that are already correct — adversarial review plus a human pass caught it.
LESSON
Spec-first beats prompt-first, measurably. Let a skeptic try to break it.
A release checklist in a doc is easy to skip under pressure — and a machine can only verify half of it; the rest is human judgment no script can prove.
APPROACH
Pluggable checks behind one interface: four AUTO checks (changelog, CI status, pinned deps, clean tagged release point) plus attestations a human signs in `release.yaml`. Exit-code driven so CI can actually block a release.
VALIDATION
90 tests · ruff-clean · v0.1.0, MIT. It gates its own releases. An adversarial pass fixed three crash-on-malformed-manifest bugs, and a security review hardened how it reads manifests and git.
FINDINGS
Every result is labeled AUTO (machine-proven) or ATTESTED (human-claimed), and the tool never upgrades one to the other — it stays honest about what it cannot verify.
LESSON
Enforce the checklist, don't just publish it. Label what's proven vs. trusted.
Python · pytest · ruff
SOURCE PRIVATE
control-roomACTIVE
TOOLING
PROBLEM
AI coding agents are usually given a vague instruction and broad write access, so a bad change can reach the working tree before anyone checks the evidence behind it.
APPROACH
Trial Mode rehearses the workflow with no account or repo. Real Mode runs three separate Codex SDK agent roles — a read-only Scout, a Builder confined to an isolated Git worktree, and a read-only Verifier — and requires explicit human approval before evidence, patch, or apply proceeds.
VALIDATION
22 tests · green CI · MIT. Submitted to OpenAI Build Week 2026, Education track.
FINDINGS
Splitting investigation, construction, and verification across roles that can't approve their own handoff catches what a single unsupervised agent would have shipped.
LESSON
Give the human every consequential gate, not just the final one — isolation without an approval boundary isn't supervision.
A convincing riso look needs real plate separation, dithering and mis-registration — and 'WASM is faster' is usually asserted, not measured.
APPROACH
Rust does the work in integer maths: a least-squares split into two ink plates, Floyd–Steinberg, Atkinson or blue-noise dithering, seeded grain, a whole-dot registration shift, and multiplied overprinting. A line-for-line JavaScript port races it on the visitor's own machine.
VALIDATION
35 Rust tests · 40 JS tests, including 1,000+ byte-for-byte comparisons of the JS port against the WASM build and a headless-Chrome run. Native Rust, wasm32 and JS hit the same golden fingerprints. MIT.
FINDINGS
Modern JS engines are good at typed-array loops, so the speed margin is real but modest and varies by device; the page measures it live instead of quoting a number. The stronger case for Rust is bit-exact determinism.
LESSON
Measure the speed claim on the user's machine, or don't make it.
A METAR (BKN025 27015KT 9999) describes the sky precisely, but to most people it reads like line noise.
APPROACH
A WebGL2 shader integrates atmospheric scattering and ray-marches each reported cloud layer as a volume; visibility becomes haze, wind moves the clouds, and the sun sits where it was at observation time. TypeScript parses the report and sets every shader input. On the live site a Cloudflare Pages Function proxies aviationweather.gov with a five-minute cache; bundled recorded reports are only a fallback, labelled RECORDED SAMPLE.
VALIDATION
196 unit tests · 23 smoke tests, 10 of them in headless Chrome and 4 against a local Wrangler Pages server. The parser is cross-checked against the Aviation Weather Center's own decoder on 400 real reports; sun elevation matches astropy to 0.03° on 12 cases; cloud cover seen from below is within 0.9 points of the reported amount across 18 reports. MIT.
FINDINGS
Coverage was only honest after calibration: seen from below, a thick layer covers more sky than any one slice of it. Cloud shapes are still procedural — a METAR gives amount and base, nothing else — and the page says which parts are data and which are style. On the deployed site the shared cache was seen working (a MISS, then a HIT); CPU time per request is still unmeasured.
LESSON
Separate the data from the style on screen, then measure the data.
A summary of legal text sounds confident and gives you nothing to check it against — and a reader that needs a paid model can't be free for every visitor.
APPROACH
Pyodide (CPython compiled to WebAssembly) runs the project's own Python in a Web Worker, so a reading is free and the text never leaves the device: a clause segmenter, then 53 hand-written rules built from 136 bounded regular expressions, with negation and near-miss checks — no AI. Every matched quote goes through the verifier written for model output — an exact substring inside the clause it labels — and drops are counted on the page. A Claude reader exists in the repo but isn't deployed.
VALIDATION
297 pytest · 33 Node tests, including headless Chrome with the strict CSP enforced (no upload during a reading) and the built bundle returning exactly what CPython returns. On 24 spot-check sentences written after the rules: 12 labelled as expected, 12 unlabelled, 0 wrong. MIT; Pyodide and CPython notices ship with the site.
FINDINGS
When a rule fires its label is usually right, but on unseen text it stays silent on about half the clauses that matter, and the page says so. The verifier proves a quote is in the text, not that the label is right. The first reading downloads a 13.6 MB Python runtime, stated before it starts.
LESSON
Check every quote mechanically, whoever wrote the reading, and say what the rules can miss.
Erosion toys often look plausible while quietly creating or losing water and rock, and their rivers are traced from noisy water depth.
APPROACH
A virtual-pipe hydraulic-erosion model (Mei et al., 2007) with conservative sediment transport that books every source and sink, in Zig with @Vector SIMD and no allocation per step. Rivers come from a separate drainage survey — Priority-Flood pit filling, D8 routing, flow accumulation — that the physics never reads.
VALIDATION
29 Zig tests · 28 Node tests, plus headless Chrome. Water and material books balance every step (a deliberate 0.1% leak fails both tests); the SIMD build matches scalar bit for bit; delta growth tracks river discharge at r = 0.80 on the 512² grid. MIT.
FINDINGS
The deltas are modest (15–20 cells of new coast) and the delta test is fragile: r = 0.54 at 256², 0.39 on another survey, and nearby parameters swung it from about −0.5 to 0.8. The README says so.
LESSON
Book every source and sink — a conservation test catches what a screenshot won't.
Cron implementations disagree about clock changes, and a job that is skipped, or runs twice, on the night the clocks change is easy to miss in review.
APPROACH
The engine is robfig/cron v3.0.1, the library Kubernetes uses to parse CronJob schedules, compiled with Go to WebAssembly, so the page shows that library's real behaviour rather than a re-implementation. For every day it compares the library's firings with a plain wall-clock reading and names each difference.
VALIDATION
34 Go tests · 41 JS tests. An equivalence test checks the fast walk against the plain library walk in all 598 tz zones: 9.15 million firings, 0 mismatches. The full exhaustive sweep has not finished, so this is sampling plus careful reasoning, not a formal proof. MIT.
FINDINGS
The tests found a quirk in Go itself: with the slim tz data built into WebAssembly, Time.ZoneBounds reports the period around 31 December of a leap year as ending before the time asked about. The engine corrects for it and a test pins it.
LESSON
Model the library production actually runs, then test the model against it.
A REPL shows only the final value, so evaluation order, scope and closures stay invisible.
APPROACH
All ClojureScript: the real Clojure reader turns text into data, a small-step evaluator reduces one redex at a time in Clojure's order, and persistent data structures keep every state, so stepping back is just indexing. Anything outside the subset is refused by name before a step runs.
VALIDATION
55 ClojureScript tests with 819 assertions, including golden caption sequences for every example and 71 hostile inputs that must end in a clear status · 11 Node and headless-Chrome tests. MIT, with EPL-1.0 and Apache-2.0 notices for the compiled bundle.
FINDINGS
It is a subset, not Clojure: no macros, lazy sequences, destructuring or loop/recur, and map and filter are eager. About 65% of the 70 KiB gzipped bundle is cljs.core. Tested in Chrome only.
LESSON
Refuse what you don't support, by name, instead of half-supporting it.
A puzzle's par is only fair if it is the true minimum — and a search that finds a good timetable doesn't prove there is no better one.
APPROACH
The rules, schedule and conflict detection are pure Elm functions of one immutable value, so undo and redo are two stacks. Exhaustive custom types (HeadOn | RearEnd | Crowded) kept the rules consistent while they changed. A Node mirror of the rules searches for each plate's par and proves it.
VALIDATION
159 elm-test · 41 Node and headless-Chrome tests. Par is proven optimal for plates I–VIII: all 15,312,644 timetables with less waiting were tried and none solves its plate. The mirror agrees with the Elm rules on 741 timetables. MIT.
FINDINGS
Plates IX–XIII have far too many timetables to try (plate IX alone about 6 × 10¹⁰), so their par is labelled heuristic and the stamp says so if you beat it. The proof runs on the mirror, not on the Elm code itself.
LESSON
Prove what you can; label the rest as a heuristic.
Browser audio code must be deterministic, never allocate, and stay safe for the listener's ears, even when fed an extreme patch or a hostile link.
APPROACH
C11 with no libc, libm or heap: 8 voices × 6 operators, 8 algorithms, per-sample smoothing of every edit, then a compressor, a −3 dBFS peak limiter and a −1 dBFS hard ceiling. zig cc builds a 14.8 KB module with no imports that runs inside an AudioWorklet; JavaScript only draws the panel.
VALIDATION
844 C checks, run under UndefinedBehaviorSanitizer and at -O2, including a 600-patch fuzz · 38 JS tests, including 22,000 hostile links. The native and WASM builds produce bit-identical samples. MIT.
FINDINGS
The voices were tuned by measuring rendered spectra and levels, not by ear; no human has listened yet. There is no oversampling, so BELL still aliases on very high notes. Tested in headless Chrome only, with the audio muted.
LESSON
Put the safety limits in the engine, then attack them with hostile input.
A tangle can't be judged by eye: the same knot shows different crossings from different angles, and an eleven-crossing mess can be the unknot.
APPROACH
Pure TypeScript finds the crossings, builds the PD code and Alexander matrix, and takes the determinant by fraction-free Bareiss elimination over BigInt polynomials, so no floating point touches a coefficient. Relaxation provably never passes the rope through itself. three.js only draws.
VALIDATION
62 unit tests · 18 end-to-end tests. Every table polynomial is recomputed from Knot Atlas PD codes. 0 bad flips in 4,800+ fuzzed crossing flips. MIT.
FINDINGS
The verdict says 'consistent with', never 'is': the Alexander polynomial can't tell a knot from its mirror image, and larger knots can share a small one's polynomial. Relaxing doesn't always untangle — the polynomial is what shows the eleven-crossing plate is the unknot.
LESSON
Compute the exact part exactly, and word the verdict no stronger than the maths.
Stroke-order practice needs specific feedback — which stroke comes first, which way it runs — not a right-or-wrong at the end.
APPROACH
A Kotlin Multiplatform core with no browser APIs compares each stroke with KanjiVG's reference on position, shape (discrete Fréchet distance) and heading, forwards and backwards; a sealed-class state machine turns that into a verdict and a plain sentence. Kotlin/JS compiles it for the web; the same tests run again on the JVM.
VALIDATION
155 tests: 75 JVM, 57 JS, 7 data, 16 headless Chrome. A whole-set sweep catches 320/320 skipped-ahead strokes and 367/367 reversed ones, and 1,279/1,280 skips with sloppy writing. MIT code; KanjiVG data under CC BY-SA 3.0.
FINDINGS
Every accuracy figure comes from simulated strokes; accuracy on real learners' handwriting is unmeasured. Some first strokes are genuinely ambiguous, because nothing is written yet to align to.
LESSON
Say where the numbers came from — synthetic strokes are not real handwriting.