What is an LLM router?
AutoRouter8 min read
An LLM router is software that sits between an application and several large language models and decides which model handles each request. Most routers aim to lower cost without losing answer quality, or to keep requests flowing when a provider fails. AutoRouter, the Mac app for Codex and Claude Code, is one kind of LLM router: it picks a model for each turn in those two apps, after the person's own rules, and uses the ChatGPT and Claude plans they already pay for first.
Several products with similar names, including LiteLLM's Auto Router, OpenRouter's Auto Router and Cursor's Auto, are made by other companies and are unrelated to AutoRouter. AutoRouter wrote this post. Claims about other products come from their own docs, linked where used and read on October 22, 2026.
What does an LLM router do?
An LLM router takes a request addressed to one model name, chooses which model or provider answers it, and returns the answer to the app. You call OpenRouter's Auto Router by sending openrouter/auto as the model, for example, and you deploy Microsoft Foundry's model router like any single model (October 22, 2026). The choice can rest on the prompt's text, how hard the task looks, price, availability, the user's rules or the user's subscriptions.
What kinds of LLM routers are there?
There are five common kinds: gateways, rule-based routers, learned or classifier routers, cascades and plan-aware routers. Many products combine two or more.
| Kind | How it picks | What it optimizes |
|---|---|---|
| Gateway | Sends a named model's request to one of several providers, by price, load or failures | Uptime and price per token |
| Rule-based | Fixed rules, such as a keyword, a mode or a context length | Predictable choices you control |
| Learned or classifier | A trained or heuristic classifier predicts which model will answer well enough | Cost at a target quality |
| Cascade | Tries a cheaper model, scores the answer, and moves up when the score is too low | Cost, with extra calls on hard requests |
| Plan-aware | Weighs which models your subscriptions already cover against per-token prices | Use of plans you already pay for |
Gateways usually keep the model you asked for and change where it runs. LiteLLM's Router adds retries, fallbacks and least-busy, latency-based or cost-based strategies, and OpenRouter favors providers without recent outages, then the lowest price (October 22, 2026).
Rule-based routers follow rules someone wrote. Claude Code's opusplan setting uses Opus in plan mode and Sonnet for execution (October 22, 2026). Rules are easy to predict, and they cover only the cases someone wrote down.
Learned and classifier routers predict which model will answer well enough. RouteLLM (June 2024) trained routers on human preference data, and its authors reported cost reductions of over 85% on MT Bench, 45% on MMLU and 35% on GSM8K against using only GPT-4, at 95% of GPT-4's performance. Hybrid LLM (ICLR 2024) reported up to 40% fewer calls to the large model with no drop in response quality. Microsoft Foundry's model router and Amazon Bedrock intelligent prompt routing are products of this kind (October 22, 2026).
Cascades try a cheaper model first. FrugalGPT (May 2023) scores each answer and calls the next model only when the score falls below a threshold; its authors reported matching GPT-4 with up to 98% lower cost on their tasks. A hard request can pass through several models before it is answered.
Plan-aware routers know which subscriptions you already pay for, such as a ChatGPT or Claude plan, and which models each one covers, and weigh that coverage against per-token prices when they pick. AutoRouter is one. In Codex, OpenAI models run on the ChatGPT plan; in Claude Code, Claude models run on the Claude plan; other models are paid per use.
What are the trade-offs of using an LLM router?
Every LLM router accepts some risk to quality, latency or caching in exchange for lower cost or better uptime.
- Quality. A misjudged request goes to a model that can't do it well. Amazon says its routing may not suit specialized use cases and depends on its training data, and Microsoft's Balanced mode accepts any model within about 1% to 2% of the best one's quality for a prompt, then picks the cheapest (October 22, 2026). For coding, see does model routing make your results worse?
- Latency. A classifier adds a step before each request, and a cascade can call several models in a row.
- Caching. Each model keeps its own prompt cache. Claude Code's docs say the request after a model switch gets no cache hits, and Anthropic bills a cache read at a tenth of the base input price or less (October 22, 2026), so frequent switching raises input cost.
- Context window. Microsoft says its router's effective context window is that of its smallest model unless you limit the pool (October 22, 2026).
- Cost. Savings depend on how much work a cheaper model can do, plus any router fee. OpenRouter's Auto Router adds no fee of its own (October 22, 2026); AutoRouter bills paid models at the provider's price plus 10% and earns nothing on plans.
Is AutoRouter the same as LiteLLM's or OpenRouter's Auto Router?
No. AutoRouter is a Mac app by Fractal Works, Inc. for Codex and Claude Code, and the products below are built by other companies for other setups.
| Name | What it is | How you use it |
|---|---|---|
| AutoRouter | Mac app that adds 20+ models to Codex and Claude Code and picks one for each turn | Pick AutoRouter in the Codex model menu, or with /model in Claude Code |
| LiteLLM Auto Router | LiteLLM proxy feature, in beta, that sorts each request into a tier mapped to a model | auto_router/complexity_router in your LiteLLM config |
| OpenRouter Auto Router | API model ID that classifies the prompt's task type and ranks models by what OpenRouter users spend on it | Send openrouter/auto as the model |
| Cursor Auto | Auto option in Cursor's model picker; on Teams and Enterprise plans, Cursor Router classifies each agent request | Choose Auto in Cursor |
For routers that work with Codex and Claude Code, see model routers for Claude Code and Codex, compared.
Where does AutoRouter fit among LLM routers?
AutoRouter is a plan-aware router that applies the person's rules first and makes every model clear a quality bar for the task before price counts. For each turn:
- If one of your rules applies, it decides the turn, for example "Use Claude Opus 5.5 for UI work."
- Otherwise AutoRouter works out how hard the turn is and only considers models that clear the quality bar for it.
- Hard work goes to the most capable model your plans cover, and to credits only when none is strong enough.
- Everything else goes to the model that costs you least, often your plan or a low-cost open model. Unclear work is never treated as a quick task.
In the landing page's illustrations, a change across six files goes to Claude Opus 5.5 on the Claude plan, and a commit message goes to a low-cost open model so the plan isn't touched. Every turn shows its model, what it cost and why, and choosing a specific model in the picker sends turns straight to it. AutoRouter sometimes misjudges how hard a task is, and you can rate the pick when it does. It works with Codex (the app and the CLI) and the Claude Code CLI on macOS 14 or later, and it is in beta. More in how AutoRouter picks a model for each turn.
Questions
Is an LLM router the same as an LLM gateway?
A gateway is the proxy that forwards requests to providers, and a router is the logic that chooses the model; many gateways include one. LiteLLM's proxy offers both load balancing and Auto Routing, while Cursor's Auto is a router inside an app.
Does an LLM router lower answer quality?
It can, whenever it sends a request to a model that can't handle it. RouteLLM's savings were measured at 95% of GPT-4's benchmark performance (July 1, 2024). Unless one of your rules names a model, AutoRouter makes every model clear a quality bar for the task before price counts, and it sends unclear work to a model that can do it well.
Does AutoRouter publish benchmarks?
No, AutoRouter doesn't publish benchmarks yet. Ratings from people and tests of each new model as it becomes available retune its routing, and the AutoRouter team reviews real routing decisions.
What does AutoRouter cost?
AutoRouter Free is $0 a month and routes across your ChatGPT and Claude plans. Pro is $20 a month with $20 of included usage each month and adds every other model; paid models are billed at the provider's price plus 10%.
Sources
- AutoRouter
- Ong et al., RouteLLM: Learning to Route LLMs with Preference Data, arXiv, June 26, 2024, revised February 23, 2025
- LMSYS: RouteLLM, July 1, 2024
- Ding et al., Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing, ICLR 2024, arXiv, April 22, 2024
- Chen, Zaharia and Zou, FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance, arXiv, May 9, 2023
- LiteLLM: Router, load balancing and fallbacks, read October 22, 2026
- LiteLLM: Auto Routing, read October 22, 2026
- LiteLLM: Auto Router with Claude Code and Claude Desktop, read October 22, 2026
- OpenRouter: Auto Router, read October 22, 2026
- OpenRouter: Provider selection, read October 22, 2026
- Cursor: Cursor Router, read October 22, 2026
- Microsoft Learn: Model router for Microsoft Foundry concepts, updated October 19, 2026, read October 22, 2026
- AWS: Understanding intelligent prompt routing in Amazon Bedrock, read October 22, 2026
- Claude Code: Model configuration, read October 22, 2026
- Claude Code: How Claude Code uses prompt caching, read October 22, 2026
- Anthropic: Prompt caching, read October 22, 2026
