# Which step of a deep research task needs the strongest model?

AutoRouter · Oct 29, 2026 · 11 min read

AutoRouter, the Mac app for Codex and Claude Code, ran a trial of DeepResearch Bench, a public benchmark of deep research tasks, on 20 of its tasks on October 28 and 29, 2026, to find which step of the task needs the strongest model, and the research step was the one that did. Claude Haiku 5.5 scored about one point below Claude Opus 5.5 when it wrote the report from Opus 5.5's research notes, on a scale where Opus 5.5 averaged 53.15, and 3.41 points below when it did the research as well. The most expensive writer tested, Claude Fable 5.1, cost 2.4 times as much as Opus 5.5 to write and scored 0.26 points lower.

This was a one-off trial with a substitute grader, so its scores compare within this study and don't belong on the benchmark's leaderboard. Each setup used a fixed pair of models with no router in between, so the trial measures each model at each step and leaves AutoRouter's own routing untested. AutoRouter wrote this post.

## How did the trial work?

The trial drew 20 tasks at random from the benchmark's 50 English tasks before anything ran, using a fixed seed, and covered 14 of its 22 topics, from finance to literature. Every call ran through Claude Code in print mode on a Claude subscription, with its built-in web search and page fetch and no other tools. Each task was researched once by Opus 5.5 and once by Haiku 5.5, and the reports came from five setups:

Every report was scored with the benchmark's RACE grader, which compares a report with the benchmark's reference report on comprehensiveness, insight, instruction following and readability, and gives a score where 50 means level with the reference. The benchmark's official grader model wasn't available to this account, so GPT-6.1 Sol graded every report, without being told which model wrote it. To measure the grader's own noise, 10 reports were scored twice: the two scores differed by 0.39 points on average and by 0.84 at most.

The pass rule was written down before the counted runs: a setup holds up if the low end of the 95% interval of its score change against Opus → Opus sits above −2 points. The method, written before any counted call, and all 140 research and writing runs, one dated line each, are published: method and results.

## How much does a cheaper writer lower the score?

A cheaper writer lowered the score by about one point in this trial. Writing from the same Opus 5.5 notes, Sonnet 5.5 scored 0.88 points below Opus 5.5 and Haiku 5.5 scored 1.04 below, and both cleared the pass rule with room to spare. Fable 5.1 came closest at 0.26 below. Giving Haiku 5.5 the research as well cost 3.41 points, and its interval falls entirely below the −2 bar.

The loss was small and steady across tasks. Sonnet 5.5 scored below Opus 5.5 on 18 of 20 tasks, and its worst task was 2.09 points down; Haiku 5.5's worst was 3.69 points down. With Haiku 5.5 doing the research, 7 of the 20 tasks lost more than 4 points.

Length explains part of Sonnet 5.5's gap. Its reports were shorter than Opus 5.5's on 19 of 20 tasks, at a median of 79% of the length, and RACE tends to reward fuller reports (correlation 0.53 between the length gap and the score gap). An exploratory fit puts its gap at equal length near −0.34 points. Haiku 5.5 wrote reports about as long as Opus 5.5's, so its gap comes from the writing itself.

## Where does a deep research report's cost go?

Research is most of the cost of a deep research report. At list prices, Opus 5.5 spent $5.27 per report on research, with about 24 web searches and 19 page fetches, and $0.488 on writing, so writing was 8.5% of the $5.76 total.

Swapping the writer cut the writing step by 58% with Sonnet 5.5 and 97% with Haiku 5.5, which comes to 5% and 8% of the whole report, because writing is a small share of a deep research task. Savings grow with the share of the work a cheaper model can take: in what common Claude Code tasks cost, measured on each model, Claude Haiku 5.5 did everyday coding tasks for about a twenty-fifth of Opus 5.5's cost per task, and every run on every model passed its check. The figures here are Claude Code's list-price estimates for each call, averaged over 20 tasks; the runs used a subscription, so no one paid these amounts per call.

## Does the most expensive writer do better?

The most expensive writer didn't beat Opus 5.5 in this trial. As of October 2026, Claude Fable 5.1 lists at $10 per million input tokens and $50 per million output tokens, against $4 and $20 for Opus 5.5 (Anthropic pricing). Writing from the same notes, Fable 5.1 scored 52.88, 0.26 points below Opus 5.5, and beat it on 4 of 20 tasks.

Its writing step cost $1.15 per report, 2.4 times Opus 5.5's $0.488, which made the whole report 11.5% more expensive. The Fable 5.1 runs were a follow-up, written down before its first counted call and run about two hours after the others with the same notes, prompts and grader.

## Do cheaper writers make more unsupported claims?

The cheaper writers made no more unsupported claims than Opus 5.5 in this trial. GPT-6.1 Sol listed every claim in each report that the research notes don't support: Opus 5.5 made 2.78 per 1,000 words, Sonnet 5.5 made 2.38 and Haiku 5.5 made 2.48. Sonnet 5.5's drop is the only one whose interval stays below zero (−1.17 to −0.01 per 1,000 words), and it had fewer on 14 of 20 tasks. Fable 5.1 made 4.22 per 1,000 words, 1.37 more than Opus 5.5 (interval 0.91 to 1.82), and fewer than Opus 5.5 on only 2 of 20 tasks.

The grader also counted citation numbers that point at the wrong source: Opus 5.5, Sonnet 5.5 and Haiku 5.5 were level at about 2 per 1,000 words, and Fable 5.1 stood out at 3.46. Both checks compare each report with its research notes, because the benchmark's own web-based citation check needs a separate scraping key.

## Why does the research step matter most?

Research set the ceiling in this trial: with the same writer, Haiku 5.5's research cost 2.37 points more than Opus 5.5's. Haiku → Haiku scored below Opus → Haiku on 19 of 20 tasks (interval −3.17 to −1.59), and the only difference between those two setups is who gathered the notes.

Haiku 5.5 wasn't lazier at research. It ran about as many web searches (22 per task against 24 for Opus 5.5), fetched more pages (27 against 19) and wrote notes of about the same size, around 30,000 characters. The difference is in what it found and kept, which a writer can't add back later.

## Does this mean you shouldn't switch models?

No. Every setup in this trial, including Opus 5.5 doing both steps, handed the research to the writer as a notes file in a separate call, so giving the writing to a different model added no handoff of its own. What moved the score was how well each model handled its step: Haiku 5.5 gave up about one point as the writer and 3.41 as the researcher.

Choosing by price alone missed in both directions. Haiku 5.5 for everything cost 93% less per report and fell well below the pass bar, while the most expensive writer, Fable 5.1, cost 11.5% more per report and scored below Opus 5.5. Opus 5.5 for both steps scored highest at $5.76 per report, and Haiku 5.5 writing up Opus 5.5's research cleared the pass bar at $5.29. The best setup depends on the step, the quality the work needs and what each step costs, which is the choice a router makes on every turn.

## What does this mean for model routing?

The trial points the same way as AutoRouter's own rules: the right model depends on the task, and the strongest model earns its price on the step where the quality is made. AutoRouter sends each task to the model that does it best, and every model has to clear a quality bar for the task before price counts at all, so a cheaper model gets work only where it holds up. Hard work goes to the strongest model your plans cover, even when a cheaper one comes close, and quick tasks go to the cheapest capable model.

The best model isn't always the most expensive, which Fable 5.1 showed at the writing step. The full order AutoRouter follows is in how AutoRouter picks a model for each turn, what an LLM router is compares other kinds of routers, and does model routing make your results worse? covers the quality question for coding.

## How do you test which model a step needs?

Test each model on the step itself, against the strongest model on the same tasks, with the bar set before you look at results. This trial followed five rules that make a comparison like this hold up:

1. Fix the pass bar first. The −2 point rule and the 20 tasks were chosen and written down before any counted run (method).
2. Compare on the same input. Every writer worked from the identical Opus 5.5 notes, so a score change belongs to the writer.
3. Measure the grader's noise. Ten reports scored twice moved 0.39 points on average, well under the one-point gaps reported here.
4. Check more than the score. Unsupported claims and wrong citations caught Fable 5.1's weakness, which its score alone hid.
5. Publish every run. All 140 runs are in results.jsonl, so anyone can check the numbers.

AutoRouter tests every new model as it becomes available, and the ratings people send on its picks retune the router; benchmarks and automatic evaluations on real coding tasks are coming soon. Anthropic recommends building your own tests from your own prompts and data (choosing a model).

## What can't this trial tell you?

• - It is small. 20 tasks from one benchmark, one run each: enough to separate a one-point loss from a three-point one, and little finer.
• - The grader is a substitute. GPT-6.1 Sol used the benchmark's official prompts, criteria and reference reports, but its scores don't compare with the public leaderboard.
• - Faithfulness is checked against the notes. A claim the notes support but the web contradicts goes uncounted.
• - RACE rewards length, and a shorter report can be the better one for a reader.
• - One harness and list prices. Every run used Claude Code's own search and fetch tools, and costs are Claude Code's list-price estimates.

## Questions

### Which model should do the research in a deep research task?

The strongest model you can afford should do the research. In this trial, Opus 5.5's research led to reports 2.37 points better than Haiku 5.5's with the same writer, and research was where most of the cost went either way.

### Is a cheaper writer worth it for deep research?

It depends on how much of the cost the writing carries. On deep research, writing was 8.5% of the cost, so Sonnet 5.5 and Haiku 5.5 saved 5% and 8% per report for about a point of score. On quick coding tasks, where a cheaper model can take the whole task, what common Claude Code tasks cost, measured on each model found Haiku 5.5 at about a twenty-fifth of Opus 5.5's cost with every run passing.

### Is this an official DeepResearch Bench result?

No. It is a trial run on 20 of the benchmark's 100 tasks with GPT-6.1 Sol in place of the official grader model, so the scores compare only within this study. AutoRouter doesn't publish benchmarks yet, and this trial tested fixed pairs of models with no router involved.

## Sources

• - DeepResearch Bench, Du et al., 2025: the tasks, reference reports and RACE grader used here
• - DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents, Du et al., arXiv, June 2025
• - Pricing, Claude Platform docs, Anthropic
• - Choosing the right model, Claude Platform docs, Anthropic
• - Method and results for this trial, AutoRouter, October 2026