{"schemaVersion":"choosely.capi.snapshot.v1","snapshot":{"id":"CAPI-V1-2026-09-23","date":"2026-09-23","status":"frozen","methodologyVersion":"1.0","verificationLabel":"Verified canonical snapshot","sourceLockDate":"2026-09-23","lastReviewedDate":"2026-10-07"},"index":{"name":"Choosely AI Progress Index","shortName":"CAPI","descriptor":"How close AI is to doing complex digital work on its own.","summary":"A conservative, evidence-backed measure of what today’s best AI can actually do — not what companies say it can do."},"pillars":[{"id":"reasoning","name":"Reasoning & Adaptation","question":"Can AI figure out difficult problems and adapt when things change?","weight":0.3,"summary":"Static novel reasoning is near the V1 ceiling, while verified interactive adaptation has advanced sharply but remains incomplete.","metrics":[{"id":"arc2","name":"ARC-AGI-2","shortName":"ARC-AGI-2","publicLabel":"Novel reasoning","publicFinding":"AI can solve most difficult unfamiliar static puzzles on the frozen semi-private surface.","displayResult":"95.0%","metric":"Semi-Private pass@2 accuracy","rawValue":95,"normalizedScore":95,"pillarWeight":0.5,"headlineWeight":0.15,"system":"GPT-6 Astra · Max","systemBoundary":"model_direct","surface":"Semi-Private Evaluation Set · 120 tasks","evaluator":"ARC Prize Foundation","evidenceStatus":"VERIFIED","verifiedAt":"2026-09-09","sourceUrl":"https://arcprize.org/results/openai-gpt-6-astra","methodologyAnchor":"arc-agi-2","note":"ARC Prize Verified Semi-Private score on the frozen ARC-AGI-2 surface."},{"id":"arc3","name":"ARC-AGI-3","shortName":"ARC-AGI-3","publicLabel":"Adapting to hidden rules","publicFinding":"AI can now solve a majority of the semi-private interactive adaptation surface under ARC Prize’s provider-neutral Standard harness.","displayResult":"62.71%","metric":"Semi-Private interactive adaptation score","rawValue":62.71,"normalizedScore":62.71,"pillarWeight":0.5,"headlineWeight":0.15,"system":"GPT-6 Astra · Max · Standard harness","systemBoundary":"model_direct","surface":"Semi-Private · 55 environments · Standard harness","evaluator":"ARC Prize Foundation","evidenceStatus":"VERIFIED","verifiedAt":"2026-09-09","sourceUrl":"https://arcprize.org/results/openai-gpt-6-astra","methodologyAnchor":"arc-agi-3","note":"Standard harness only: ARC Prize describes it as the minimal provider-neutral interface. Provider Adapter results that preserve opaque reasoning state are excluded from CAPI."}]},{"id":"work","name":"Real-World Work","question":"Can AI reliably finish useful work from start to finish?","weight":0.3,"summary":"The weakest pillar: accepted professional deliverables remain much harder than polished demos suggest.","metrics":[{"id":"rli","name":"Remote Labor Index","shortName":"RLI","publicLabel":"Professional work","publicFinding":"Most professional projects still do not reach a client-acceptable finish without human help.","displayResult":"20.83%","metric":"Automation Rate","rawValue":20.83,"normalizedScore":20.83,"pillarWeight":0.55,"headlineWeight":0.165,"system":"GPT-6 Astra","systemBoundary":"agent_system","surface":"230 private professional projects","evaluator":"Scale Labs / RLI operator","evidenceStatus":"VERIFIED","verifiedAt":"2026-09-23","sourceUrl":"https://labs.scale.com/leaderboard/rli","methodologyAnchor":"rli","note":"A project counts only when independent reviewers judge the deliverable client-acceptable."},{"id":"ale","name":"Agents’ Last Exam","shortName":"ALE","publicLabel":"Broad agent tasks","publicFinding":"AI completes just under one in three broad professional agent tasks in full on the frozen ALE-V1 surface.","displayResult":"31.6% full pass rate","metric":"Overall Full Pass Rate","rawValue":31.6,"normalizedScore":31.6,"pillarWeight":0.45,"headlineWeight":0.135,"system":"Claude Code + Claude Opus 5 · High","systemBoundary":"agent_system","surface":"ALE-V1-FULL-2026-08-30 · 152 tasks","evaluator":"Agents’ Last Exam","evidenceStatus":"VERIFIED","verifiedAt":"2026-09-09","sourceUrl":"https://agents-last-exam.org/leaderboard","methodologyAnchor":"ale","note":"Overall full pass rate only on ALE-V1; partial-credit Score and Best-per-task composites are excluded."}]},{"id":"autonomy","name":"Autonomy & Agency","question":"Can AI keep working on its own without constant human help?","weight":0.4,"summary":"AI can reach into longer expert tasks, but strict end-to-end computer-workflow completion stays low.","metrics":[{"id":"metr","name":"METR Time Horizon 1.1","shortName":"METR","publicLabel":"Longer autonomous work","publicFinding":"AI can now tackle tasks that take human experts hours, but reliability falls as work gets longer.","displayResult":"~11h 59m at 50% success","metric":"50% success horizon, normalized to a 40-hour target","rawValue":719,"rawUnit":"human-expert task minutes","normalizedScore":84.51,"pillarWeight":0.4,"headlineWeight":0.16,"system":"Claude Opus 4.6","systemBoundary":"agent_system","surface":"TH1.1 · approximately 11h 59m","evaluator":"METR","evidenceStatus":"VERIFIED","verifiedAt":"2026-08-30","sourceUrl":"https://metr.org/time-horizons/","methodologyAnchor":"metr","note":"This is task difficulty, not literal unattended runtime; uncertainty is substantial near the suite’s edge."},{"id":"osworld2","name":"OSWorld 2.0","shortName":"OSWorld 2.0","publicLabel":"Computer use","publicFinding":"AI often makes partial progress across computer workflows but rarely finishes them cleanly.","displayResult":"20.6% binary accuracy","metric":"Binary Accuracy","rawValue":20.6,"normalizedScore":20.6,"pillarWeight":0.6,"headlineWeight":0.24,"system":"Claude Opus 4.8 · Max · Batched tool","systemBoundary":"agent_system","surface":"v2026.06.24 · 108 tasks · 500-step budget","evaluator":"OSWorld 2.0 benchmark team","evidenceStatus":"VERIFIED","verifiedAt":"2026-08-30","sourceUrl":"https://osworld-v2.xlang.ai/","methodologyAnchor":"osworld-2","note":"Strict binary completion is scored; the 54.8% partial score is display-only."}]}],"liveEdge":[{"metricId":"arc3","label":"Interactive adaptation jumps","detail":"ARC-AGI-3 Semi-Private rises to 62.71% under the provider-neutral Standard harness.","tone":"high"},{"metricId":"arc2","label":"Static reasoning nears the V1 ceiling","detail":"ARC-AGI-2 reaches 95.0% on the frozen Semi-Private surface.","tone":"high"},{"metricId":"metr","label":"Longer tasks, wider uncertainty","detail":"AI can attempt work that would take human experts almost 12 hours, but uncertainty rises as tasks get longer.","tone":"high"}],"gaps":[{"metricId":"rli","title":"Deliver client-acceptable work reliably","detail":"20.83% of the frozen professional project set is automated to the acceptance threshold."},{"metricId":"osworld2","title":"Finish long computer workflows cleanly","detail":"Strict binary completion is 20.6%, despite materially higher partial progress."},{"metricId":"ale","title":"Pass broad professional agent tasks in full","detail":"Overall ALE-V1 full-pass performance is 31.6%; most tasks still fail the complete-outcome threshold."}],"evidenceQueues":{"underVerification":[{"title":"New benchmark results","detail":"Checked before they can change the Index","tag":"CHECKING"},{"title":"Company performance claims","detail":"Not counted without independent evidence","tag":"EXCLUDED"},{"title":"Updated test versions","detail":"Reviewed for a fair like-for-like comparison","tag":"REVIEW"}],"open":[{"title":"A reliable history of progress","detail":"This first snapshot is where the series begins","tag":"OPEN"},{"title":"Human impact signals","detail":"Not published until defensible evidence exists","tag":"OPEN"}]},"capabilityLimits":[{"metricId":"arc3","icon":"◇","title":"Make interactive adaptation dependable","detail":"ARC-AGI-3 has advanced sharply, but 62.71% still leaves substantial failure on hidden-rule environments.","valueType":"metric","label":"ARC-AGI-3"},{"metricId":"rli","icon":"▦","title":"Deliver client-acceptable work","detail":"Most professional projects still need human work before independent reviewers call the result client-ready.","valueType":"metric","label":"RLI automation"},{"metricId":"osworld2","icon":"⌘","title":"Finish computer workflows cleanly","detail":"AI often makes useful progress on computer tasks but fails to complete every required step.","valueType":"metric","label":"OSWorld binary"},{"metricId":"ale","icon":"✓","title":"Pass broad agent tasks in full","detail":"Broad agent tasks only count when the whole job is finished correctly; partial progress does not pass.","valueType":"metric","label":"ALE full pass"},{"metricId":"osworld2","icon":"↯","title":"Turn progress into completion","detail":"OSWorld partial progress is 54.8%, but strict completion is only 20.6%.","valueType":"osworld_completion_gap","label":"completion gap"},{"metricId":"arc3","icon":"∞","title":"Close the static-to-interactive reasoning gap","detail":"The ARC2-to-ARC3 spread is now 32.29 points, far narrower than the launch snapshot but still material.","valueType":"reasoning_balance","label":"remaining balance"},{"metricId":"metr","icon":"◷","title":"Measure longer tasks reliably","detail":"The 11h 59m horizon sits near the current suite’s measurement-quality edge.","valueType":"caution","label":"suite-edge caution"},{"metricId":"rli","icon":"◎","title":"Reach the real-work milestone","detail":"RLI remains 29.17 points short of Choosely’s 50% project-automation milestone.","valueType":"rli_milestone","label":"of milestone"}],"capabilityHistory":[{"date":"Mar 2023","title":"GPT-4 launches","detail":"Broad multimodal reasoning advances","state":"reached","sourceUrl":"https://openai.com/index/gpt-4-research/"},{"date":"Jun 2023","title":"Function calling","detail":"Models begin using external tools","state":"reached","sourceUrl":"https://openai.com/index/function-calling-and-other-api-updates/"},{"date":"Mar 2024","title":"Claude 3 Opus","detail":"Long context and robust analysis","state":"reached","sourceUrl":"https://www.anthropic.com/news/claude-3-family"},{"date":"May 2024","title":"GPT-4o","detail":"Real-time multimodal interaction","state":"reached-warm","sourceUrl":"https://openai.com/index/hello-gpt-4o/"},{"date":"2025–26","title":"Agentic systems","detail":"Autonomous step-by-step workflows","state":"reached-warm","sourceUrl":"/ai-progress/methodology#architecture"},{"date":"Aug 2026","title":"AI Progress Index V1.0","detail":"First canonical measurement","state":"reached-warm","sourceUrl":"/ai-progress/methodology#calculation"},{"date":"Next","title":"Reliable autonomy","detail":"The work ahead","state":"open","sourceUrl":"#open-gaps"}],"milestones":[{"date":"2026-03","title":"Long-horizon measurement enters the edge of the suite","detail":"METR’s usable public reference reaches roughly 11h 59m at 50% success, with explicit uncertainty warnings.","sourceUrl":"https://metr.org/notes/2026-03-20-impact-of-modelling-assumptions-on-time-horizon-results/"},{"date":"2026-06","title":"Desktop work gets a stricter binary surface","detail":"OSWorld 2.0 v2026.06.24 tests 108 workflows with a 500-step budget.","sourceUrl":"https://osworld-v2.xlang.ai/"},{"date":"2026-08","title":"A frozen evidence basket becomes reproducible","detail":"Six exact benchmark surfaces, 10,000 basis points and a 24% concentration ceiling produce the first CAPI snapshot.","sourceUrl":"/ai-progress/methodology"},{"date":"2026-09","title":"The World Ahead becomes a consequence layer","detail":"The approved 2026 edition visualises a plausible world shaped by verified capability — it does not forecast it.","sourceUrl":"#world-ahead"}],"supportingMilestones":[{"title":"Professional deliverable benchmark","status":"REACHED","detail":"The human-anchored GDPval-AA v2 Elo level has been exceeded by frontier systems. This is benchmark-specific, not universal professional-work parity.","sourceUrl":"https://artificialanalysis.ai/evaluations/gdpval-aa"}],"worldAhead":{"editionId":"WA-L00-2026","title":"Choosely World Ahead 2026","status":"frozen","approvalStatus":"APPROVED","frozenAt":"2026-09-04","desktopSha256":"0e44a42e981caaee1cbe4c2d03ab5d1d921f373e804e92b54256cc77da7ada77","mobileSha256":"e66f4d69899564f7069fc09d51ff380b22b3659224d1fd194c5cecfc4fbcc423","positioning":"An evolving glimpse of the future made plausible by today’s verified AI capabilities.","bridge":"The Index tells you how far capability has moved. The World Ahead lets you feel why that might matter.","humanSignals":[],"changes":[]},"calculated":{"exactScore":49.825050000000005,"displayScore":49.8,"pillars":[{"id":"reasoning","name":"Reasoning & Adaptation","exactScore":78.855,"displayScore":78.9,"weight":0.3},{"id":"work","name":"Real-World Work","exactScore":25.6765,"displayScore":25.7,"weight":0.3},{"id":"autonomy","name":"Autonomy & Agency","exactScore":46.164,"displayScore":46.2,"weight":0.4}]},"distributions":{"json":"https://choosely.ai/ai-progress/data.json","csv":"https://choosely.ai/ai-progress/data.csv","methodology":"https://choosely.ai/ai-progress/methodology"}}