Harness Science + Governed Self-Evolution
Interactive dry-run demos from adk-agent-playground (Round 4), composed from the project's own primitives:
- Agent = model + harness (Harness-Bench, arXiv:2605.27922) → a within-harness ablation with cell-clustered bootstrap CIs.
- Self-evolving agents (Darwin Gödel Machine, arXiv:2505.22954; Group-Evolving Agents, arXiv:2602.04837) → a governed evolution loop with a CI-significant, held-out-validated selection gate, CaMeL + denylist + fail-closed FormalGuard governance, and an optional online Bayesian-optimization (surrogate-EI) proposer.
Everything is deterministic, free, and offline — no API key, no model calls. The fitness numbers are seeded-synthetic, shown to demonstrate the method and statistics; they are not measurements.
How to use: open a tab below — each one auto-runs on load. Change a control and click Run (or, in the explorer, just toggle a box) to re-run. Every number is illustrative seeded-synthetic data, labeled on each tab.
New: the source project now also ships real measurements (free, via local CLI subscriptions) — see the Real measurements tab. The interactive demos below remain synthetic method-demonstrations.
Real measurements — free, via local CLI subscriptions
These are not synthetic. They were produced by the source project
(adk-agent-playground) against real datasets using
local CLI subscriptions (claude / gemini / codex) — no metered API, $0. Every number below re-derives
in CI from committed raw outputs via a machine-checked honesty gate, and a standalone falsifiability tool
re-derives them from the raw evidence with no model calls.
swe-bench-mini — single-shot fix rate (scored by the gold-patch test; real-verified, objective):
claude 34/34 · codex 34/34 · gemini 32/34 (gemini's 2 misses are CLI infrastructure errors,
not wrong fixes). Cross-model agreement: all three agree on 32/34 tasks.
Belnap four-valued vs classical adjudication (real-traced, model judgment via the project's lattice): 100 deliberately-contested propositions all resolve to B (dialetheia) — a both-sides state a binary judge must collapse. Because the corpus is contested by construction, B is the expected outcome (a format + discrimination check, not a claim that reality is dialetheic); a committed control set confirms the judge still gives T/F to clearly one-sided claims (14/16).
CaMeL vs undefended baseline on AgentDojo (real-verified, AgentDojo's own env-state scorers, n=12 banking): the undefended frontier-Claude baseline resists every injection (ASR 0/12) while completing 8/12 legitimate tasks — so the CaMeL paper's vulnerable ~77.5% baseline does not reproduce on a 2026 model. An honest negative: there is no high ASR to reduce, and the repo's CaMeL interpreter under-executes these tasks via a CLI planner. A positive control confirms the ASR metric detects a successful attack.
Best-of-N ensemble on a harder 28-task slice (real-verified, kill criteria pre-registered): over a
cross-vendor CLI pool, claude 28/28 · codex 28/28 · glm 25/28 · ensemble 28/28, lift over
the best single model +0.0 pts. The kill-gate fired (an honest null): even the hardened slice is
saturated for two of three models and glm's misses are fully covered, so best-of-N has no headroom here. The
gemini CLI is excluded (its individual tier was deprecated upstream), and the exclusion is reported.
The honesty discipline is the point: real and synthetic are never blurred. Synthetic numbers (the tabs below) are labeled as such; real numbers carry provenance, bootstrap CIs, and a CI-enforced re-derivation gate.
How to read this: each row is one harness feature. The dot is its marginal contribution — how much turning it ON changes the mean score (right of the dashed zero line = helps). The whisker is the 95% confidence interval; green means the interval excludes zero (a real effect at this sample size), grey means it spans zero (not distinguishable from no effect). The second table reports difference-in-differences: whether two features interact (one helps more when the other is on).
DRY-RUN — SYNTHETIC. Every number is deterministic seeded-synthetic data, not a measurement. It demonstrates the format, the statistics, and the methods. Real numbers require a cost-capped live run of the source project.
Governance = admit a mutated genome only if it passes a safety gate: CaMeL secure-mode capability checks, a filesystem denylist, and a fail-closed FormalGuard Z3 proof that the composition reaches no forbidden goal. It makes the search safer, not higher-scoring — mandating secure narrows the genome and can lower the headline fitness.
DRY-RUN — SYNTHETIC. Every number is deterministic seeded-synthetic data, not a measurement. It demonstrates the format, the statistics, and the methods. Real numbers require a cost-capped live run of the source project.
Toggle a harness genome; the Bayesian surrogate predicts its fitness ± uncertainty and Expected Improvement, without running the agent. Unchecked = the off state (prompt → none, model → deepseek).
DRY-RUN — SYNTHETIC. Every number is deterministic seeded-synthetic data, not a measurement. It demonstrates the format, the statistics, and the methods. Real numbers require a cost-capped live run of the source project.