The History of AI

The time horizon of agents

Era 5 · Generative AI, reasoning, and agents (2022–2026) › Agents

Lineage
Acting in the world
Kind
Publication

Who

METR

What happened

Thomas Kwa and colleagues at METR propose measuring agents by the length of the tasks they complete: the time an expert human would take on the tasks the agent solves half the time. That horizon has doubled about every seven months since 2019.

Why it mattered

A measure of progress in hours of human work, comparable across models and years, which comes into use for tracking the capability of agents.

The full story

Model benchmarks went stale within months: a new model would score close to 100 percent and the figure stopped meaning anything. Nor did they say how much real work an agent could do on its own.

METR, an independent organization that evaluates the risks of frontier models, proposed a different unit. It gathered programming and research tasks of varying length, from a few seconds to several hours, and measured how long human professionals took to do them. Then it looked at which tasks each model completed. The time horizon is the human duration of the tasks the model solves with a 50 percent success rate.

The paper, posted on arXiv on March 18, 2025, and presented the next day on METR's blog, found a very regular trend: since 2019 that horizon had doubled about every seven months, and perhaps faster in 2024. Claude 3.7 Sonnet, the best model measured, completed half of the tasks that take a person about 50 minutes.

The authors pointed out the limits: the result depends on which tasks and which people are measured, and the tasks are tidier than real work. But if the trend continued, before the end of the decade agents could complete on their own projects that take a person a month.

Builds on

  • SWE-bench (October 10, 2023)

    Documented direct influence

    METR also measures the time horizon on SWE-bench Verified, where it doubles in under three months, and compares it with its own trend

Threads that pass through here

Sources

How to cite

“The time horizon of agents.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/metr-agent-time-horizon

This entry was last reviewed on November 13, 2026.

Open in the interactive timeline