Brida
Most AI benchmarks ask a model to solve a large task and then collapse the outcome into one score. ReflexBench starts somewhere else: small, repeated decisions that software and agents need to make quickly, cheaply and with a stable typed contract.
Today we are releasing ReflexBench v1.0.0, an open benchmark and reproducible evaluation harness for System One models and typed decision engines.
The benchmark is designed around three bounded decision shapes:
- Binary / Noul — a calibrated yes/no decision, including abstention where the engine supports it.
- Choice — select from a bounded set of options.
- Score — return a bounded semantic score that software can consume directly.
The goal is not to declare a universal “best model.” The goal is to make the important trade-offs measurable: correctness, calibration, multilingual consistency, option-order robustness, supported cardinality, failure behavior and the value of deterministic workflow policy around the model.
Why System One AI needs its own evaluation layer
Large reasoning models are powerful, but many production decisions do not need a long reasoning trace. An agent deciding whether to wake up, a pipeline classifying an incoming event, or a product selecting one of a small set of actions often needs a fast bounded judgment instead.
That is the space we call System One AI: immediate semantic decisions with explicit output contracts.
A useful benchmark for this space has to measure more than raw accuracy. A model that is right 90% of the time but badly calibrated, fragile to option order, or unable to express abstention can be harder to operate than a slightly less accurate model with stable probabilities and predictable failure modes.
ReflexBench therefore publishes the dimensions separately instead of hiding them inside one leaderboard number.
What ReflexBench v1 measures
The v1 protocol includes:
- same-corpus engine evaluation;
- probability calibration and confidence behavior;
- Binary/Noul, Choice and Score contracts;
- English and Spanish lanes;
- option-order robustness;
- Choice cardinality and capability limits;
- machine-readable receipts;
- retained failures rather than silently deleted rows;
- same-response policy evaluation, where the model response is held constant and only the deterministic Reflex policy changes;
- blind workflow gates that measure whether the full decision system actually improves a downstream task.
The harness and methodology are public. The v1 protocol is frozen so that future engine and checkpoint results remain comparable.
The first public results
One result is especially important for how we think about Brida Reflex.
On the public same-response product fixtures, the underlying Jev response is held fixed. Applying the deterministic Reflex Core policy moves the measured result from 96.4% to 100.0%: 4 rescues, 0 harms and 0 additional model calls.
That is not a claim that a harness can magically improve every model. It is evidence for a narrower idea: in bounded decision systems, deterministic policy around a probabilistic engine can remove known failure modes without paying for another inference.
We then tested the idea in a separate blind workflow replication. Gate v2 completed all 150 rows and moved from 76.7% to 82.0%, an improvement of +5.3 percentage points, with a paired bootstrap 95% interval of +1.3 to +10.0 points. The gate recorded 10 rescues and 2 harms.
We also publish the failed first attempt. Gate v1 is retained as a formal FAIL because one provider 503 left the run at 149/150 completion. On the completed rows it moved from 45.0% to 64.4%, but the incomplete run is not promoted to a passing benchmark result.
That failure is part of the benchmark record by design.
Same corpus, real receipts
ReflexBench also includes a public hard cohort for engine comparison. In our run, TypeSafe Jev reached 73.0% on the exact 111-row cohort. An upstream kshetrajna12/reflex configuration using Qwen3.5-2B reached 41.4% on that same local cohort.
Those numbers are useful, but they are not a global ranking. Different engines have different deployment surfaces, latency characteristics, calibration behavior and capability limits. ReflexBench keeps those dimensions visible and publishes the receipts needed to inspect them.
That is also why we do not collapse hosted and local latency into one “speed winner.” Hardware-normalized local inference and hosted API latency are different measurements.
Search and retrieval belong upstream
Web search can improve decision quality dramatically when the task depends on fresh or missing evidence. But adding search silently to one model and not another makes the model benchmark meaningless.
Our integration pattern is explicit:
web / search / retrieval
↓
provenance + freshness filters
↓
bounded evidence
↓
System One model
↓
Reflex policy / reviewThis lets us evaluate retrieval quality separately from decision quality. A future retrieval-augmented lane can be added as a named protocol extension without pretending that “internet access” is intrinsic model intelligence.
Open methodology, private operational advantage
The public release contains the benchmark contract, methodology, synthetic public corpora, harness, receipts and release checks.
Brida’s hosted control plane, routing logic, provider credentials, tenant systems, private corpora and future hidden gates remain private. We want the evaluation protocol to be inspectable and useful outside Brida without turning operational infrastructure into benchmark theater.
The public Reflex ecosystem now has three complementary surfaces:
- Brida Reflex — the open registry and decision recipes.
- Brida SDK — the public integration surface.
- ReflexBench — the benchmark, methodology and reproducible receipts.
v1 is a historical protocol, not a moving target
ReflexBench v1 is now frozen. New model or checkpoint results can be added in v1.x when they use the unchanged protocol. Material changes to the core corpus, scoring or evaluation methodology will become ReflexBench v2.
That distinction matters. A benchmark becomes more valuable when old results remain interpretable after newer models arrive.
The repository includes the complete results, methodology, benchmark card, receipts and release-manifest verification.