The time horizon of agents
Era 5 · Generative AI, reasoning, and agents (2022–2026) › Agents
- Lineage
- Acting in the world
- Kind
- Publication
Who
METR
What happened
Thomas Kwa and colleagues at METR propose measuring agents by the length of the tasks they complete: the time an expert human would take on the tasks the agent solves half the time. That horizon has doubled about every seven months since 2019.
Why it mattered
A measure of progress in hours of human work, comparable across models and years, which comes into use for tracking the capability of agents.
The full story
Model benchmarks went stale within months: a new model would score close to 100 percent and the figure stopped meaning anything. Nor did they say how much real work an agent could do on its own.
METR, an independent organization that evaluates the risks of frontier models, proposed a different unit. It gathered programming and research tasks of varying length, from a few seconds to several hours, and measured how long human professionals took to do them. Then it looked at which tasks each model completed. The time horizon is the human duration of the tasks the model solves with a 50 percent success rate.
The paper, posted on arXiv on March 18, 2025, and presented the next day on METR's blog, found a very regular trend: since 2019 that horizon had doubled about every seven months, and perhaps faster in 2024. Claude 3.7 Sonnet, the best model measured, completed half of the tasks that take a person about 50 minutes.
The authors pointed out the limits: the result depends on which tasks and which people are measured, and the tasks are tidier than real work. But if the trend continued, before the end of the decade agents could complete on their own projects that take a person a month.
Builds on
SWE-bench (October 10, 2023)
Documented direct influence
METR also measures the time horizon on SWE-bench Verified, where it doubles in under three months, and compares it with its own trend
Threads that pass through here
- From answering to acting (step 7 of 9)
Sources
Original source
- Measuring AI Ability to Complete Long Tasks (arXiv:2503.14499) (opens in a new tab). Thomas Kwa et al., 2025
Scholarship and context
How to cite
“The time horizon of agents.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/metr-agent-time-horizon
This entry was last reviewed on November 13, 2026.