CHOOSELYMethodology V1.0 · Frozen 14 October 2026
AI Progress Index methodology.
Every score, source and rule.
A public, reproducible account of what CAPI measures, what it excludes and how six verified observations become one conservative score.
01 · Architecture
What the Index measures.
CAPI measures demonstrated frontier capability on difficult, externally evaluated digital work. It is not a forecast, a model popularity ranking, a product leaderboard or a claim about general intelligence.
Reasoning & Adaptation
Static novel reasoning is near the V1 ceiling, while verified interactive adaptation has advanced sharply but remains incomplete.
78.9 / 100Real-World Work
The weakest pillar: accepted professional deliverables remain much harder than polished demos suggest.
28.9 / 100Autonomy & Agency
AI can reach into longer expert tasks, but strict end-to-end computer-workflow completion stays low.
46.2 / 10002 · Deterministic calculation
From six scores to 50.8.
0.50(95.00) + 0.50(62.71) = 78.8550.55(21.25) + 0.45(38.20) = 28.87750.40(84.51) + 0.60(20.60) = 46.1640.30(78.855) + 0.30(28.8775) + 0.40(46.164) = 50.7854 → 50.8The source file is parsed into a typed schema, then the same calculator supplies the webpage, JSON, CSV and snapshot share card. Display rounding happens only after the exact composite is calculated.
03 · Evidence policy
Claims need a surface, not a slogan.
Headline eligible
Exact benchmark surface, metric, system boundary, evaluator and primary source are resolved.
Visible, not scored
A relevant result exists, but at least one required identity or provenance field remains unresolved.
Research queue
A question or claim worth tracking without enough evidence for a public capability statement.
Vendor claims do not enter CAPI unless they match the frozen evaluation surface. Living benchmarks require an exact snapshot and a series-break review when their task corpus, grader or scoring definition changes.
04 · Frozen metric register
Every input, in the open.
ARC-AGI-2
- Scored metric
- Semi-Private pass@2 accuracy
- Frontier system
- GPT-6 Astra · Max
- Frozen surface
- Semi-Private Evaluation Set · 120 tasks
- System boundary
- model_direct
- Evaluator
- ARC Prize Foundation
- Headline weight
- 15%
ARC Prize Verified Semi-Private score on the frozen ARC-AGI-2 surface.
Open primary source ↗ARC-AGI-3
- Scored metric
- Semi-Private interactive adaptation score
- Frontier system
- GPT-6 Astra · Max · Standard harness
- Frozen surface
- Semi-Private · 55 environments · Standard harness
- System boundary
- model_direct
- Evaluator
- ARC Prize Foundation
- Headline weight
- 15%
Standard harness only: ARC Prize describes it as the minimal provider-neutral interface. Provider Adapter results that preserve opaque reasoning state are excluded from CAPI.
Open primary source ↗Remote Labor Index
- Scored metric
- Automation Rate
- Frontier system
- Opus 5.5
- Frozen surface
- 230 private professional projects
- System boundary
- agent_system
- Evaluator
- Scale Labs / RLI operator
- Headline weight
- 16.5%
A project counts only when independent reviewers judge the deliverable client-acceptable.
Open primary source ↗Agents’ Last Exam
- Scored metric
- Overall Full Pass Rate
- Frontier system
- Claude Code + Claude Opus 5.5 · Max
- Frozen surface
- ALE-V1-FULL-2026-08-30 · 152 tasks
- System boundary
- agent_system
- Evaluator
- Agents’ Last Exam
- Headline weight
- 13.5%
Overall full pass rate only on ALE-V1; partial-credit Score and Best-per-task composites are excluded.
Open primary source ↗METR Time Horizon 1.1
- Scored metric
- 50% success horizon, normalized to a 40-hour target
- Frontier system
- Claude Opus 4.6
- Frozen surface
- TH1.1 · approximately 11h 59m
- System boundary
- agent_system
- Evaluator
- METR
- Headline weight
- 16%
This is task difficulty, not literal unattended runtime; uncertainty is substantial near the suite’s edge.
Open primary source ↗OSWorld 2.0
- Scored metric
- Binary Accuracy
- Frontier system
- Claude Opus 4.8 · Max · Batched tool
- Frozen surface
- v2026.06.24 · 108 tasks · 500-step budget
- System boundary
- agent_system
- Evaluator
- OSWorld 2.0 benchmark team
- Headline weight
- 24%
Strict binary completion is scored; the 54.8% partial score is display-only.
Open primary source ↗05 · Governance
A frozen baseline, with explicit change rules.
Freeze record
Source lock, system boundaries, Grade-A evidence, the ALE manifest, ARC2 provenance, independent calculation, basis-point totals and concentration guard all passed.
ARC2 provenance
The Verified 95 score is Semi-Private. Public task details shown on the same result page are diagnostic and are not used to infer the scored surface.
Revision rule
Corrections are recorded; benchmark version changes are adjudicated; incompatible surfaces create a series break instead of a silent splice.
No synthetic history
This is the first canonical snapshot. CAPI history begins here; earlier capability events appear only as sourced milestones.
Stable source register
Snapshot: CAPI-V1-2026-10-14 · source lock: 2026-10-14 · methodology: V1.0