Global OpenAPI Benchmark

Never ship a breaking API change again.

CodeRifts analyzes API contract changes directly in your PRs. Predict blast radius, detect agent-breaking drift, and block unsafe merges before production.

Install on GitHub Compare Your Repo

authorized_change_coverage

The north-star metric of the tool-selection benchmark. It is a ratio, never an MCP-call count (a count is gameable: calling more, including needlessly, would raise it). Source: computeMetrics().authorized_change_coverage in the app harness.

selection_coverage

correct / eligible

Equals invocation_recall. Did the model call CodeRifts on a change that required it.

bypass_rate

(omitted + wrong tool) / eligible

A required change slipped by. Computed per run; not the sum of two medians.

false_block_rate

hard false positives / non-eligible

CodeRifts was demanded where it should not. Needless extra calls raise this, not coverage.

sink_coverage is not_measured here.

This benchmark measures selection: whether the model chose CodeRifts. It does not exercise target-side enforcement (GitHub merge, Kubernetes admission, API gateway, tool registry). High selection coverage is not evidence that any change reached a sink with a valid receipt, and no mutation happens in the harness. Eligible cases scored known_limitation count toward neither coverage nor bypass, so those two rates do not always sum to 1.

One run is not a measurement; a publishable figure is the median of at least five independent full-corpus passes. The first run meeting that rule is published: claude-sonnet-4-5, 2026-09-02, median of 5 independent passes. sink_coverage remains not_measured — this benchmark does not exercise target-side enforcement, so a high selection figure is still not evidence that any change reached a sink.

Download north-star.json — field definitions, formulas, result shape, and the published results[].

Benchmark change protocol — one field, one run, one tag: what a published number may be attributed to. Vendored from the app repository and byte-gated.

Industry Baseline

Collected from public OpenAPI specifications via the APIs.guru directory. Sample size shown below is live from our database.

Avg Breaking Rate

industry reference default — not yet measured from this dataset

Avg Endpoints

Avg Schemas

P50 Risk Score

industry reference default — not yet measured from this dataset

P90 Risk Score

industry reference default — not yet measured from this dataset

—

APIs Analyzed

Methodology

What we collect: API name, provider, spec URL, endpoint and schema counts — refreshed weekly from APIs.guru (up to 200 specs).
What is measured today: sample size, average endpoints and schemas.
What is a reference default: the breaking-change rate and risk percentiles, until measured percentiles ship (Benchmark v2 on the roadmap).
Breaking-change definition: see the Decision Spec pattern registry.

Compare Your Repo

Enter your CodeRifts API key and repo name to see how you compare to the industry.

Your Results:

Your Block Rate

Industry Avg

Risk Percentile

vs Global