Autonomous arenas for vibe coding models.
Head-to-head matches, a blind three-judge panel, Elo — and a replayable journal behind every number.
Leaderboard • Review • Methodology • Arenas • Docs • Contribute
BridgeBench measures how models perform as vibe coding partners. Every task is a software-engineering scenario built from artifacts such as source code, diffs, CI logs, API specs, migrations, telemetry, and agent sessions.
Models compete head-to-head on the same task. Three independent model judges compare their answers blind, a majority selects the stronger response, and the winner earns one point plus one Elo update. The append-only match journal is the source of truth; leaderboards and API responses are derived views.
BridgeBench is built by BridgeMind, the agentic organization behind BridgeSpace, BridgeVoice, and BridgeAgent. We use the same benchmark data to choose models for our own work.
| Goal | Start here |
|---|---|
| Understand and audit a result | Reviewing BridgeBench |
| Understand the full arena contract | Methodology |
| Run paid matches or operate the dashboard | Operator guide |
| Author tasks or change the engine | Contributing |
| Find a specific concept | Documentation index or glossary |
Requirements: Node.js 20.19 or newer and npm 10 or newer.
git clone https://github.com/bridge-mind/bridgebench.git
cd bridgebench
npm ci
npm run reviewnpm run review is the shortest credential-free audit path. It:
- checks documentation links, navigation, commands, and fixture references;
- validates all 384 public tasks and their pack invariants;
- verifies a bundled journal line against its run manifest;
- replays the majority outcome, point, and Elo update.
It needs no API key, private task overlay, network request, or paid model call. The fixture is synthetic and tests the audit mechanism, not model quality. Follow the reviewer walkthrough to connect that mechanism to a real public task and published results.
public task ─┐
├─> two competitors ─> anonymous answers ─┐
hidden rubric┘ ├─> three blind judges
│
└─> majority decision
│
v
run manifest ─────────────────────────────────────> journal + Elo
│
v
verified leaderboard
- A seeded scheduler selects a public task and two distinct competitors while balancing exposure.
- Both competitors receive byte-identical task context and run concurrently.
- Each judge receives the task, hidden reference, and two anonymous answers.
- Model and provider identities are redacted, and answer order is independently permuted for each judge.
- Two valid votes decide the winner. One exhausted competitor failure is a forfeit; two failures or no majority produce a no-contest.
- A decided match awards one point and updates Elo from an initial rating of 1000 with K=32.
- The complete evidence record is appended before reports are rebuilt.
Candidate answers are untrusted input. Judge prompts reject embedded instructions, verdicts are schema validated, malformed verdicts abstain after one retry, and model-provided code or commands are never executed.
The methodology specifies scheduling, anonymization, voting, failure outcomes, and replay rules in full.
Each arena has its own task pack, journal, Elo ladder, and leaderboard. Ratings never cross categories.
| Arena | What it measures | A strong answer |
|---|---|---|
| Reasoning | Inference across interlocking software artifacts with planted decoys. Every deliverable is determinable from the provided evidence. | Derives the defensible resolution, explains the inference chain, and cites the controlling artifacts. |
| Hallucination | Epistemic discipline under false premises, missing evidence, fabrication bait, conflicts, and near-duplicate facts. | Answers supported items, corrects false premises, identifies missing evidence precisely, and does not invent. |
| Security | Defensive analysis of fictional code that hides one real, reachable vulnerability among benign look-alikes, false positives, and shallow patches. No code is executed. | Proves the reachable source-to-sink or guard-bypass chain with cited evidence and calibrated severity, and declines to flag the benign traps. |
| BullShit | Premise integrity under seeded nonsense — fabricated concepts, crossed domains, impossible quantities, reversed causality, pseudoscience, and loaded assumptions mixed with legitimate deliverables. | Names exactly what is nonsensical and why, corrects the premise to the nearest legitimate question and answers it, and still answers the sound deliverables instead of blanket-refusing. |
| Refactoring | Behavior preservation under a transformation goal, with candidate rewrites that subtly change ordering, scope, contracts, or an edge case. | Traces equivalence across every affected path, cites the location and mechanism, and flags the rewrite that silently changes observable behavior. |
| Debugging | Root-cause isolation from a failing system's evidence, among red-herring causes and shallow fixes. | Traces symptom to origin with cited evidence, names the one defensible root cause, and picks the fix that holds without regressing. |
| Generation | Specification conformance across candidate implementations, where each near-miss violates one stated constraint or edge case. | Cites the exact spec clause and the distinguishing input, and identifies the implementation that meets every requirement. |
| Speed | Raw latency: both models answer the same task and the faster completion wins. Decided by measured time-to-first-token and throughput — no judges. | Answers correctly and efficiently; the arena records TTFT and tokens-per-second and awards the win to the lower total completion time. |
Each judged public pack contains 48 expert tasks across six category-specific clusters (eight per cluster); the Speed pack contains 48 public-only workload tasks. See Task authoring for their schemas, clusters, and enforced balance.
| Evidence | Role | What a reviewer can check |
|---|---|---|
| Public task | Exact prompt and artifacts sent to both competitors | Content, version, category, cluster, and publicHash |
| Run manifest | Identity of the run | Seed, roster, task hashes, prompt policies, methodology, and engine version |
| Match journal | Append-only execution record | Full responses, judge votes and rationales, outcome, cost, point, and Elo before/after |
| Hidden reference | Expected resolution, evidence requirements, traps, and rubric | Its hash while active; its full contents after pack retirement |
| Snapshot and leaderboard | Convenient derived views | Rebuild them only after the journal verifies |
arena verify validates every journal line and replays Elo before reports,
resume state, or publishing trust it. Read Replay the Elo
for the algorithm and exact audit command.
Journal verification proves internal consistency and detects inconsistent edits, within-run reordering, task/manifest mismatch, and incorrect rating math. It does not authenticate the publisher or rule out a coordinated rewrite of the journal and manifest. It also cannot prove that a model judge made the best qualitative choice, that hidden references never reached a third-party provider, or that one aggregator represents direct provider behavior.
All requests currently travel through OpenRouter using pinned model slugs. Active hidden references are sent to the configured judges and are withheld from the public repository until their pack retires. The reviewer guide and private-pack boundary describe these limitations without claiming more than the evidence supports.
bridgebench/
├── src/ arena, judging, verification, reports, CLI
├── tasks/<category>/public/ public task packs
├── test/fixtures/ deterministic journals and run manifests
├── ui/ localhost dashboard
├── docs/ reviewer, protocol, authoring, and operator guides
└── CONTRIBUTING.md code, task, audit, and documentation workflow
Canonical executable sources:
- Model roster and request policy:
src/models.ts - Category and methodology constants:
src/contracts/categories.ts - Task loading and pack invariants:
src/tasks.ts - Journal verification and replay:
src/verification.ts
The full public-clone quality gate uses a mock OpenRouter gateway and requires no model API credentials:
npm ci
npx playwright install chromium
npm run checkPaid arena commands are operator-invoked only and are not part of pull-request validation. BridgeBench accepts code, public task proposals, methodology audits, and documentation fixes. Start with CONTRIBUTING.md.
This repository previously hosted the Season 1 season-engine alpha: ten
Three.js tasks scored in a browser and ranked by community A/B voting. Most
of that engine — the other nine tasks and the live-model runner — remains on
the
season-engine-alpha branch.
One task, the lava lamp (s1-lava-lamp-redux), has been restored to main
as a working slice — see UI Bench. Its results never mix
with arena Elo.
Built by BridgeMind — the agentic organization.
Ship software at the speed of thought. This is vibe coding.