Pull to refresh
Logo
Daily Brief
Following Why Ranks Sign Up
AI agents quadruple their score on a real freelance-work benchmark

AI agents quadruple their score on a real freelance-work benchmark

New Capabilities

The Remote Labor Index jumps from 2.5% to 16.1% of paid projects completed at professional quality in under eight months

3 days ago: Frontier score more than quadruples

Overview

Eight months ago, the best AI agent could finish 2.5% of real freelance jobs well enough for a paying client to accept. On July 6, the Center for AI Safety reported that a new model now clears 16.1%.

The number is small, but the slope is steep. The test uses 240 real paid projects, and human professionals earned about $144,000 doing them. AI agents are still failing most of the work. They are failing less of it, fast.

Why it matters

If AI agents keep closing this gap at the current pace, freelance work in design, video, and data analysis is the first labor market to feel it.

Questions about this story

No questions yet — be the first to ask.

Play on this story Voices Debate Predict Higher or Lower

Key Indicators

16.1%
Top automation rate
Share of paid projects the best agent, Fable 5, completed at professional quality.
2.5%
Rate 8 months earlier
The best agent's score when the benchmark launched in October 2025.
240
Real projects tested
Paid freelance jobs across 23 categories, each judged against professional work.
$144K
Human earnings on the same work
Combined amount freelancers were paid for the 240 projects.
84%
Projects agents still fail
Even the top model falls short of client-acceptable quality on most work.

Voices

Curated perspectives — historical figures and your fellow readers.

Ever wondered what historical figures would say about today's headlines?

Sign up to generate historical perspectives on this story.

Log in Sign Up

Play

Exploring all sides of a story is often best achieved with Play.

Log in to play. Track your picks, climb the leaderboards. Log in Sign Up
Predict 3 ways this could play out. Contrarian picks score more — points lock when the scenario resolves. Log in to play
Higher or Lower Two numbers from this story. Guess which is bigger. 5 rounds to set a streak. Log in to play
Connections Sixteen names from the news. Find the four hidden groups of four. Log in to play

People Involved

Dan Hendrycks
Dan Hendrycks
Leading the group that runs the benchmark
 Bing Liu
Bing Liu
Co-led the benchmark's design and evaluation

Organizations Involved

Center for AI Safety
Center for AI Safety
Nonprofit Research Organization
Runs and publishes the Remote Labor Index

A San Francisco nonprofit that studies risks from advanced AI and builds tests to measure what AI systems can do.
Scale AI
Scale AI
AI Data and Evaluation Company
Partner that runs the human grading

A company that supplies data and evaluation services to AI developers, providing the human graders behind the benchmark.

Timeline

October 2025 July 2026

2 events Latest: 3 days ago

  1. Frontier score more than quadruples

    Latest Research

    New results show Fable 5 at 16.1%, Opus 4.8 at 8.3%, and GPT-5.5 at 6.3%. The top score has more than quadrupled from the previous leader's 4.2% in under eight months.

  2. Benchmark launches with a 2.5% ceiling

    Research

    The Center for AI Safety and Scale AI release the Remote Labor Index. The best agent, Manus, completes 2.5% of the 240 paid projects at professional quality.

Historical Context

3 moments from history that rhyme with this story — and how they unfolded.

September 2012

ImageNet and the deep-learning breakout (2012)

A neural network called AlexNet cut the error rate on the ImageNet image-recognition contest by roughly 10 percentage points in one year. The benchmark had moved slowly before. Then it moved fast.

Then

Research funding and talent poured into deep learning within months.

Now

The approach became the base for modern computer vision, from phone cameras to self-driving car perception.

Why this matters now

A single benchmark can flip from slow crawl to steep climb. The Remote Labor Index's jump from 2.5% to 16.1% has the same early shape, though the outcome is not yet settled.

November 2020

DeepMind's protein-folding leap on CASP (2020)

AlphaFold scored high enough on the CASP protein-structure contest that organizers called the 50-year problem largely solved. Prior systems had plateaued for years.

Then

Structural biology labs began using predicted structures almost immediately.

Now

DeepMind released structures for most known proteins, speeding drug and disease research worldwide.

Why this matters now

A trusted, hard-to-game benchmark turned a research milestone into real-world use. The Remote Labor Index aims for the same credibility by grading against paid human work.

2015 to present

Self-driving 90% problem (2015 onward)

Autonomous-driving demos quickly handled most road situations, and companies predicted robotaxis within a few years. The last fraction of hard cases proved far slower to solve.

Then

Billions in investment chased a near-term rollout that kept slipping.

Now

Deployment came, but narrowly, city by city, more than a decade after the early hype.

Why this matters now

Fast early gains do not guarantee a fast finish. The messy final projects on the Remote Labor Index could resist automation the way rare road cases did.

Sources

(4)
Center for AI Safety Center for AI Safety / Scale AI arXiv HumanProgress