# Which effort level is a fair baseline for router savings?

Authors: AutoRouter
Author profile: https://autorouter.com/about
Published: 2026-11-05T15:45:00.000Z
Reading time: 5 min
Canonical: https://autorouter.com/blog/router-savings-effort-baseline
Part of: What is an LLM router? (https://autorouter.com/blog/what-is-an-llm-router.md)
Publisher: AutoRouter, the Mac app for Codex and Claude Code (https://autorouter.com)

> A fair baseline for a router's savings is the cheapest effort level that reaches the model's top score on the same benchmark. On DeepSWE, GPT-6 Astra reaches its top score of 74% at xhigh for $4.43 per task, while max costs a reported $7.50 for a score within the same error range. A router costing $3.00 per task looks 60% cheaper than Astra at max and 32% cheaper than Astra at xhigh.

## Key facts

- On the DeepSWE leaderboard, GPT-6 Astra's best setting is xhigh, scoring 74% (±3 points) at an average of $4.43 per task over 113 tasks ([DeepSWE](https://deepswe.datacurve.ai/), read November 4, 2026).
- Kingy AI reports Astra at max scoring 73.2% for $7.50 per task on DeepSWE 1.1, 69% more than xhigh for no higher score ([Kingy AI](https://kingy.ai/blog/gpt-6-astra-vs-gpt-6-1-sol/), read November 4, 2026).
- A router costing $3.00 per task looks 60% cheaper than GPT-6 Astra at max and 32% cheaper than Astra at xhigh, with nothing changed but the baseline.
- SWE-Serve reports Claude Opus 5 going from 74% at high effort to 75% at max while mean cost per task rose 76%, from $9.89 to $17.40 ([arXiv](https://arxiv.org/abs/2609.26777)).

A fair baseline for a router's savings is the cheapest effort level that reaches the model's top score on the same benchmark. For GPT-6 Astra on DeepSWE, that is xhigh at $4.43 per task; max costs a reported $7.50 per task for a score within the same error range. [AutoRouter, the Mac app for Codex and Claude Code](https://autorouter.com), wrote this post as a reading of results others published. No router ran in it, and AutoRouter doesn't publish benchmarks yet.

## Why does effort change the baseline?

Effort sets how much a model reasons before it answers, so the same model can cost far more per task at one setting than at another. Anthropic describes `max` as its highest capability with no constraints on token spending, and for Claude Sonnet 5.5 and Claude Haiku 5.5 it advises using `xhigh` or `max` only where your evals show a quality gain ([Anthropic's effort guide](https://platform.claude.com/docs/en/build-with-claude/effort), read November 4, 2026). A baseline at max carries the most spend a model can produce, whether or not that spend buys a better score.

## What does DeepSWE show for GPT-6 Astra?

On DeepSWE, GPT-6 Astra scores the same at high, xhigh and max within the benchmark's error range, and max costs the most. The leaderboard shows each model at its best setting and lists Astra at xhigh, with 74% (±3 points) at an average of $4.43 per task over 113 tasks ([DeepSWE leaderboard](https://deepswe.datacurve.ai/), read November 4, 2026). The high and max rows are reported by [Kingy AI](https://kingy.ai/blog/gpt-6-astra-vs-gpt-6-1-sol/) from DeepSWE 1.1 results; DeepSWE's own page doesn't show them.

| GPT-6 Astra setting | DeepSWE score   | Average cost per task | Source               |
| ------------------- | --------------- | --------------------- | -------------------- |
| high                | 73.2%           | $3.92                 | Reported by Kingy AI |
| xhigh               | 74% (±3 points) | $4.43                 | DeepSWE leaderboard  |
| max                 | 73.2%           | $7.50                 | Reported by Kingy AI |

Moving from xhigh to max adds $3.07 per task, 69% more, with no gain in score. Moving from high to max nearly doubles the cost, from $3.92 to $7.50, for the same reported 73.2%.

## How much does the baseline change a savings figure?

The baseline alone can nearly double a router's savings figure. Take a router that costs $3.00 per task on DeepSWE, a made-up figure for the arithmetic:

| Baseline             | Baseline cost per task | Savings for a $3.00 router |
| -------------------- | ---------------------- | -------------------------- |
| GPT-6 Astra at max   | $7.50                  | 60%                        |
| GPT-6 Astra at xhigh | $4.43                  | 32%                        |
| GPT-6 Astra at high  | $3.92                  | 23%                        |

Only the baseline changed between those rows. Astra at xhigh is the fair baseline here because it is the cheapest setting that reaches Astra's top score, so a router measured against it has to match a model at its best without credit for spend that bought nothing.

## Does max effort pay off for Claude Opus 5?

On a different benchmark, max bought Claude Opus 5 one point for 76% more spend. SWE-Serve, a separate paper and benchmark, reports Opus 5 moving from 74% at high to 75% at max while mean cost per task rose from $9.89 to $17.40 ([SWE-Serve, arXiv](https://arxiv.org/abs/2609.26777)). The two benchmarks measure different tasks, so their numbers don't combine, but both show max costing far more for little or no gain in score.

## How should a router state its savings?

Routers of every kind quote savings, from cascades to plan-aware routers ([What is an LLM router?](/blog/what-is-an-llm-router) compares them), and four habits make a figure checkable:

1. Name the baseline model and its effort level in the same sentence as the figure.
2. Use the cheapest setting that reaches the top score on a named benchmark, with the date of the run.
3. Report the router's score next to its cost, so readers see that it reached the same score.
4. Say when the numbers were measured, because leaderboards change as new models arrive.

AutoRouter's routing starts from the same idea: "The best model isn't always the most expensive." Every model has to clear a quality bar for the task before price counts, and hard work goes to the strongest model your plans cover, even when a cheaper one comes close. [How AutoRouter picks a model for each turn](/blog/how-autorouter-picks-a-model) walks through the order it follows.

## What does this comparison not show?

This comparison covers one model on one benchmark of 113 tasks, plus one model on a second benchmark. DeepSWE reports ±3 points of run-to-run variation for Astra, so its high, xhigh and max scores fall within the same range, and the high and max rows come from a secondary report. It shows that max cost more than xhigh for Astra on DeepSWE without a higher score. It doesn't show which setting fits your own tasks, and since no router ran in it, it says nothing about how well any router picks models.

## Questions

### Is xhigh always the fair baseline?

No. The fair baseline is the cheapest setting that reaches the top score on the benchmark a figure cites, and for another model or benchmark that may be high, medium or max.

### Why not use high as the baseline, since it costs less?

The rule takes the cheapest setting that reaches the top score, and on DeepSWE's board that is xhigh at 74%. High's reported 73.2% sits within the ±3-point range, so a comparison against high is also defensible when the figure says so. Either is far closer to fair than max.

### Does max effort ever pay off?

It can, on tasks where it raises the score enough to matter. Anthropic advises using `xhigh` or `max` only where your evals show a quality gain, so test the higher setting on your own tasks before paying for it ([Anthropic](https://platform.claude.com/docs/en/build-with-claude/effort)).

### Should a router be measured against the cheapest model?

A cheap model makes a poor baseline when it fails tasks the router solves, because the comparison then mixes cost with lower quality. Compare against the cheapest setting of a model that reaches the top score, and show the router's score and cost beside it.

## Sources

- [DeepSWE leaderboard](https://deepswe.datacurve.ai/), read November 4, 2026
- [Kingy AI: GPT-6 Astra vs GPT-6.1 Sol](https://kingy.ai/blog/gpt-6-astra-vs-gpt-6-1-sol/), secondary report of DeepSWE 1.1 results, read November 4, 2026
- [Anthropic: Effort](https://platform.claude.com/docs/en/build-with-claude/effort), read November 4, 2026
- [SWE-Serve: Benchmarking Agentic Engineering For Production Inference Serving](https://arxiv.org/abs/2609.26777), arXiv

## Read next

- [Does model routing make your results worse?](https://autorouter.com/blog/does-model-routing-hurt-quality.md): Model routing makes results worse when it sends work to a model that can't do it. In its October 7, 2026 launch post, Anthropic reported 39.2% for Claude Haiku 5.5 on Terminal-Bench 4.0, a test of long multi-step terminal work, against 70.6% for Claude Sonnet 5.5. AutoRouter, the Mac app for Codex and Claude Code, keeps quality up by making every model clear a quality bar for the task before price counts, sending hard work to the strongest model your plans cover and never treating unclear work as a quick task.
- [How AutoRouter picks a model for each turn](https://autorouter.com/blog/how-autorouter-picks-a-model.md): AutoRouter, the Mac app for Codex and Claude Code, picks a model for each turn in order: your rules first, then how hard the turn is, then only the models that clear the quality bar for it. Hard work goes to the most capable model your ChatGPT or Claude plans cover, and to credits only when none is strong enough. Everything else goes to the model that costs you least, which is often your plan or a low-cost open model.
- [How do you test a model router change before it ships?](https://autorouter.com/blog/how-to-test-a-routing-change.md): Test a routing change by running real, reviewed requests through the same code path production uses, rules included, and grading each pick against the models a reviewer accepted. A test that skips the rules can pass while the shipped policy breaks one. AutoRouter, the Mac app for Codex and Claude Code, checks its routing changes against reviewed cases that include picks people rated down.

Tags: [Model routing](https://autorouter.com/blog/tags/model-routing.md), [Costs](https://autorouter.com/blog/tags/costs.md), [Models](https://autorouter.com/blog/tags/models.md)

Drafted with AI from the sources linked above, then checked and edited by the AutoRouter team.