// a miniature IP-regression platform
A DAG flow runner, EDA-style log parsing, waivers, and run-to-run diff — the core loop of a chip-design regression product, rebuilt small enough to read in an afternoon and honest enough to trust in CI.
In chip design, a nightly regression runs dozens of tools across a
dependency graph, each spewing gigabyte logs where a tool can exit
0 and still have failed. What a team actually needs to know
isn't "did tonight pass" — it's what broke that
was working yesterday. GAUNTLET rebuilds that loop end to end.
A tool exits 0 but its log holds an unwaived ERROR → the step still FAILs. Exit codes under-report; the log is the source of record.
A known failure becomes WAIVED — never silently green. It stays visible in every report with an owner and a JIRA reason.
Run-to-run, GAUNTLET reports NEW FAILURES, FIXED, and STILL FAILING — and exits non-zero on regressions, so CI can gate on it.
A flow is a YAML DAG. The scheduler submits a step only when every dependency has passed, runs it as a real subprocess in its own process group, then parses the log against a rulepack and applies waivers. Here's the smoke flow — lint gates two sims that run in parallel, then synth.
Workers spend their lives blocked in subprocess.wait(), where the GIL is released — a ThreadPoolExecutor gives real parallelism with none of the ceremony.
Steps launch in their own process group. On timeout we signal the group, grace it, then hard-kill — so a shell wrapper can't die while its simulator lives on.
When a dependency fails, dependents go SKIPPED — and the diff blames the upstream step that actually broke, not the ones it starved.
Logs are classified line by line against compiled regex rulepacks. Real EDA logs run to gigabytes; reading one whole is how you fall over in production.
Same seed, same flow — but a flake injects an overflow in u_fifo_tx.
The waiver only matches u_fifo_rx, so it doesn't fire: the step
FAILs. GAUNTLET doesn't just report the new run —
it reports what changed.
| PASS | lint_rtl | 4 warnings |
| WAIVED | sim_unit | 1 error waived |
| PASS | sim_integration | 1 warning |
| PASS | synth_quick | clean |
| PASS | lint_rtl | 4 warnings |
| FAIL | sim_unit | SIM-ERR-007 · u_fifo_tx |
| PASS | sim_integration | 1 warning |
| SKIPPED | synth_quick | upstream failure |
No plugins, no async, no web server in the core — just the behaviors a regression platform actually needs, each one deliberate.
A step is submitted only when every dependency is PASS or WAIVED. Failures cascade SKIPPED transitively; --keep-going lets independent branches finish. Ordering is proven by tests, not vibes.
Per-tool regex rulepacks compile once at load. A bad pattern is a config error at startup — exit 2 — never a surprise mid-run.
rule-id + substring match, each carrying an owner and reason. Precise, not blanket — the same rule with a different signature still fails.
history, report, and diff all work from the DB alone — long after the raw logs are cleaned up.
report <run> --html writes one self-contained page — step summary, per-step log links, violation tables — with zero new dependencies.
A step that only passes on retry is marked flaky: it satisfies its dependents, but every report says so. A flaky pass is information, not a pass.
Clone it, install the one dependency, and watch a clean run, a regression, and the diff that catches it.
GAUNTLET is one project. There are 18 more (and a few jet engines) over at charanreddy.dev. If you're hiring, collaborating, or just curious how the diff works — the door's open.