HWE-Bench
8.15%
best model tested, fully correct
How it works · the thesis
AI that engineers a circuit, not AI that draws one.
Ask a language model for a schematic and you get a drawing that looks right. Typmax inverts the loop: the model is the planner, deterministic code is the engineer, and the datasheet is the authority.
Agent (tool calls)
└─▶ HDIR design doc
└─▶ compiler ──▶ .kicad_sch
│
├─▶ rules engine
│ ▲ catalog (part records)
│ ▲ interface defs (rules as data)
│
└─▶ validation report
The evidence
The verifier is the product. The model is a commodity front-end.
In the arXiv preprints: on HWE-Bench (arXiv:2603.18102) the best model tested gets 8.15% of schematics fully correct. NetlistBench watches multi-turn SPICE-netlist editing collapse from 80% at three dependent edits to 26% at fifteen. And PCBSchemaGen, holding the model fixed and adding a verifier, moves the same backbone from 41.9% to 81.3%. The third number decided the architecture.
So the model never touches KiCad files, never holds design state, and never recalls an engineering rule from memory. It proposes small, typed changes to a structured design document. Everything that decides whether the board works is code, data and arithmetic — and every one of them arrives with its source.
NetlistBench
80 → 26%
over 3 to 15 dependent edits
PCBSchemaGen
41.9 → 81.3%
same backbone, verifier added
Non-negotiables
Nine principles we don't trade away.
01
No bare numbers
Every physical value carries min/typ/max, units and test conditions. Arithmetic goes through a unit engine. A bare float in a record is a bug.
02
Rules are data
Formulas and constants live in interface definitions with their authority cited. A protocol constant is never hardcoded — MCP73831 programs at 1000 V / RPROG, TP4056 at 1100–1200.
03
Provenance is mechanical trust
Fields are tagged datasheet / vendor_db / llm_extracted / inferred / human. Report trust degrades automatically. An llm_extracted record is never presented as verified.
04
Capability ≠ binding
The catalog says what a pin can do; the design says what it is doing. The engine checks that join on every validation — including ESP32-style mux matrices.
05
Typed mutations are the only write path
30 typed ops with optimistic concurrency. Whole-design regeneration by a model is impossible by construction. Every batch is followed by full re-validation.
06
Nets are derived, never authored
Pin-level nets come from buses, connections and rails at compile time. Net-identity drift is structurally impossible.
07
Findings show their work
Computed intermediates, assumptions cited, unverified inputs listed, source authority named, plus a machine-actionable fix hint.
08
KiCad compatibility is load-bearing
Pin types are KiCad's twelve-value enum verbatim. Export is always ungated: you own your files, in your tool, from day one.
09
Risk domains force human review
LiPo charging, mains, motor drive. Deterministic checks are necessary, not sufficient, there. This never gets softened.
The artifact
A KiCad project, compiled — not drawn.
The compiler emits record-true symbols, label connectivity and a block hierarchy into one KiCad project, with path-derived UUIDs and an independent S-expression cross-check. KiCad's own ERC runs on it in CI.
.kicad_schschematic
.kicad_proproject settings
netlistcross-checked vs derived nets
BOMreal part numbers
validation reportthe dossier
Being honest: making that schematic pleasant to read at a glance is what we are building now. Generated symbols and motif-assigned designators still look like what they are.
gps-tracker-v0.kicad_sch · compiled · producer-recorded fixture
Want the version of this with the numbers in it?
Join the waitlist and we'll send the technical write-up as it's published, plus the weekly changelog.