A letter, with footnotes

Dear TJ

You think this is hype. That's a defensible position, and most of what gets written about AI deserves it. So here is the argument without the adjectives: every number below links to where it came from, every claim is dated, and the case against comes before the conclusion — at full strength, not as a strawman.

Compiled 25 September 2026 · — · skip to the case against

01

One measurement, tracked the same way since 2019

Most AI charts measure something that flatters AI. This one doesn't: METR gives human experts real software tasks, times them, then asks how long a task an AI model can finish on its own — measured at the length where it succeeds half the time.

Task length an AI agent completes unaided, 50% of the time

Each dot is a model on release date; the bar is METR's own 95% confidence interval. The vertical axis is logarithmic — each gridline is ten times the one below. Drawn directly from METR's published data file.
— GPT-2, February 2019
— frontier model, April 2026

Since 2023 that length has doubled roughly every 129 days (METR's confidence interval: 104–158 days). Fitting a line to their released points independently gives 125 days — close enough that the trend line above is faithful to their data rather than to anyone's narrative.

What this is not. It is software tasks, not all work. The most recent points have wide error bars — the April 2026 estimate ranges from 8 hours to over two days, and you should read it as "somewhere in that range", not as seventeen hours. And a 50% success rate is not a thing you can hand a job to. Hold that thought; it comes back in §3.

02

The shape of the work changes before the headcount does

The interesting evidence isn't "AI does your job". It's that the unit of work is moving from a conversation you steer to a task you hand over and check later.

Writing a blog post in a chat window

13 rounds of back-and-forth

The same deliverable handed to a coding agent

1 human prompt

Median conversation producing a blog post, Anthropic's own traffic, 10 April – 10 June 2026. Same output, two different jobs for the human: one is steering, the other is commissioning and reviewing.

That shift is what "agentic" actually means in practice, and it is why the skill that gets scarce isn't prompting. It's specification and review — being able to say precisely what good looks like, and to tell whether you got it.

Who actually gets faster

The best field evidence we have is a peer-reviewed study of 5,179 customer-support agents in the Quarterly Journal of Economics. Access to an AI assistant raised issues resolved per hour by 15% on average — but the average hides the finding that matters:

Change in issues resolved per hour, by worker skill level.

The gain is concentrated almost entirely in the least experienced workers. For the most experienced, it is close to nothing. That is not a story about machines replacing people. It's a story about the value of experience being partially copied into a tool — which is a bigger deal for how work is organised than a headcount number would be.

And in the labour data

Stanford's Digital Economy Lab tracks payroll records for more than 26 million US workers, updated through June 2026. Their finding is deliberately unexciting in one direction and pointed in the other: no evidence of widespread, economy-wide displacement — but employment of 22-to-25-year-olds in the most AI-exposed occupations now sits 19% below where it would be had it tracked their less-exposed peers. It shows up as reduced hiring, not as layoffs.

This is observational, not an experiment — AI exposure correlates with other things about an occupation. It is the right size of claim to make from it: a visible dent in entry-level hiring, not a jobs apocalypse.

03

The strongest case against everything above

If I were arguing your side, TJ, this is what I'd lead with. None of it is fringe, and I don't think any of it is wrong.

  1. A controlled trial found AI made experienced developers slower.

    METR — the same lab as the chart above — ran a randomised controlled trial. 16 experienced open-source developers, 246 real tasks in repositories they knew well. With AI tools they took 19% longer. They had predicted AI would make them 24% faster, and afterwards still believed it had made them 20% faster. They were wrong in both direction and magnitude, about their own work, while doing it.

    Why I don't think this settles it: it is 16 developers on early-2025 tools, in codebases where they were already expert — precisely the population the QJE study found gains least. It is strong evidence that self-reported productivity is worthless, and weak evidence about the trend. But it should make you distrust every vendor testimonial you ever read, including the ones I'd otherwise cite.

  2. Most corporate AI pilots produce nothing measurable.

    An MIT working paper put the figure at roughly 95% of generative-AI pilots showing no measurable effect on profit and loss. Meanwhile 88% of organisations report using AI somewhere, but fewer than 10% have scaled it in any single business function, and agentic deployment is still in the single digits across most functions.

    Why I don't think this settles it: the 95% is from 52 interviews and 153 survey responses and has not been peer reviewed — I've cited it because it's the strongest version of your argument, not because it's rigorous. But the direction is corroborated by better sources, and it is genuinely damning about today. It says the capability exists and the organisational capacity to use it does not. Those are different claims, and only one of them is about the technology.

  3. Agents are unreliable in exactly the way that matters.

    Sierra — a company that sells AI agents — published a benchmark of realistic customer-service knowledge work. The best frontier model solved 25.5% of tasks on the first attempt, and only 9.3% reliably across four runs. Consistency, not capability, is the wall.

    Why I don't think this settles it: I don't have a rebuttal to this one. It's the best objection on the list, and it's why "agentic AI will do your job next year" is a bad prediction. What I'd say is narrower: a 9.3% reliable pass rate on a hard new benchmark in March 2026 is where SWE-bench scores were a couple of years before they saturated. That's a reason to watch the slope, not a reason to be comfortable.

  4. The benchmarks themselves are partly broken.

    OpenAI published a post titled "Why SWE-bench Verified no longer measures frontier coding capabilities" and stopped reporting it. When a lab retires the benchmark it was winning, you should discount benchmark-based arguments — including some of mine.

    Why I don't think this settles it: it's the reason I built §1 around METR rather than a leaderboard. METR's tasks are timed against human experts and held out, which is a harder thing to contaminate. But you're right to treat any single number as soft.

  5. The famous success story partly reversed.

    Klarna announced in February 2024 that its AI assistant was handling two-thirds of customer service chats — 2.3 million conversations in a month, the work of 700 agents. By 2025 the CEO was hiring humans back, saying quality had suffered.

    Why I don't think this settles it: because of where it landed. Klarna didn't turn the AI off — it still handles the routine majority, with humans on disputes, fraud and distress. The lesson isn't "AI failed". It's that full replacement failed and triage worked, which is a description of a changed job, not an unchanged one.

Why unreliable agents fail at long jobs — it's just multiplication

This is the mechanism behind objection 3, and it's arithmetic rather than a claim, so check it yourself. If each step of a task succeeds independently with probability p, a task of n steps succeeds with probability pn.

Whole task succeeds 59.9% of the time

A 95%-reliable step is an excellent component and a hopeless 20-step worker. This is why the useful question is not "how smart is it" but "how long a chain can it hold" — which is precisely what the chart in §1 measures.

The same arithmetic as a landscape — drag to turn it. Height and colour are the chance the whole task finishes; the two ridges are where that chance is 50% and 90%. The marker is wherever the sliders are.

Nothing here is measured or modelled — it is pn, and you can check any point on it with a calculator. Note how fast the 90% ridge climbs: a twenty-step job that works nine times out of ten needs 99.5% per step. That is the wall agentic systems are actually up against, and it is the sceptic's strongest ground.

04

What I actually claim

Narrower than the headlines, and harder to dismiss.

Not: AI will replace your job next year.

The reliability data says no. A system that passes 9.3% of realistic tasks consistently cannot be handed a role.

Rather: the unit of delegation is growing, fast and measurably.

Under a minute in 2019, hours in 2026, doubling every four months since 2023 on a measurement designed not to flatter. Even if that halves in pace, the direction holds.

And: the constraint has moved from capability to verification.

The bottleneck in every honest deployment story above is the same — checking the output, handling the exceptions, knowing when it's wrong. That's a description of a job, and it's not the job most people currently have.

Which is why: the effect shows up in hiring first.

A 19% relative fall in entry-level employment in exposed occupations, through reduced hiring rather than layoffs, is exactly what "the bottom rung of the ladder gets automated" looks like in payroll data before it looks like anything else.

Two more things that are simply true

The price of a given capability is collapsing. Epoch AI measured the cost of reaching a fixed performance threshold and found declines between 9× and 900× per year depending on the task — around 40×/year for PhD-level science questions, around 200×/year for general knowledge. Things that are uneconomic at today's price are a rounding error at next year's.

And the evaluations are moving toward real work. OpenAI's GDPval is 1,320 tasks drawn from 44 occupations across the nine largest sectors of US GDP, written by professionals averaging 14 years in the job and scored by head-to-head comparison against those professionals' deliverables. It's vendor-built and it measures one-shot deliverables rather than whole jobs — but the fact that it can be built at all is the point.

05

What would change my mind

An argument you can't lose isn't an argument. Here is what I'd accept as evidence I'm wrong, written down in advance so you can hold me to it.

That's the whole argument. Not that the robots are coming — that the length of a task you can hand over has gone from seconds to hours in seven years, that it is still doubling, that the binding constraint is now verification rather than capability, and that this is already visible in who gets hired. You can think the timeline is slower than the boosters say — I do — and still conclude the shape of knowledge work changes inside your career.

— with love, and receipts

✳

Sources

Every one, with what's wrong with it. Check any of them; that's the point.

    Method. The chart in §1 is generated from METR's published data file, not transcribed by hand. Where a figure could not be traced to a primary or otherwise checkable source it was removed rather than rounded or hedged — a few claims that would have helped the argument are missing for that reason, including a widely-repeated benchmark statistic I could not verify at its source. Figures are current as of 25 September 2026 and will age; the dates are there so you can tell how fast.