Model Routing for LLMs, Sending Each Request to the Right Model

What model routing actually is

Routing is just the decision of which model to call, made per request instead of once for the whole app. That is the entire idea.

The reason it matters is the price spread. A frontier model can cost more than ten times what a budget model costs for the same tokens, so any request you can safely move down is a large saving repeated across your whole volume. The full spread across providers is in how to reduce your OpenAI costs.

Three simple routing strategies

You do not need anything clever to capture most of the value. Three signals do the heavy lifting.

• Route by task type. Send summaries, tags, and drafts to a budget model, and send code, legal, or anything high stakes to a frontier model. The task tells you how much brain the request needs.
• Route by user plan. Free users get the cheap model, paid users get the strong one. This turns model quality into a plan feature and protects your margin on the tier that pays you least.
• Route by current spend. When a user crosses their budget, drop them to a cheaper model for the rest of the period. This is the cap triggered downgrade from model downgrade on a budget.

A real example architecture

Combine the three into one small function that takes what you already know about the request and returns a model name. Nothing here needs a database call beyond the spend you are already tracking.

// one router, three signals: spend, then task, then plan
function routeModel({ feature, plan, spent, cap }) {
  if (spent >= cap) return 'gpt-4o-mini'                 // over budget: downgrade
  if (feature === 'code' || feature === 'legal') return 'gpt-4o' // needs frontier
  if (plan === 'free') return 'gpt-4o-mini'              // free tier stays cheap
  return 'gpt-4o'                                        // sensible default
}

const model = routeModel({
  feature,
  plan: user.plan,
  spent: await getMonthlySpend(user.id),
  cap: capFor(user.plan),
})

const res = await openai.chat.completions.create({ model, messages })

That is a complete, useful router. It reads top to bottom, anyone on your team can change a rule, and it captures the large wins without a single extra model call. Compare the quality of the two models you are choosing between in GPT-5 vs GPT-4o-mini before you wire the rules.

The tradeoff: routing power versus maintenance

Every rule you add is a rule you now own. That is the real cost of routing, and it is easy to forget when you are excited about saving money.

The tempting next step is a classifier router: call a small model to judge how hard each request is, then route on that. Sometimes it pays off, but it adds latency to every request, adds its own token cost, and introduces a component that can be wrong in ways that are hard to debug. A static rule is boring and predictable, which is usually what you want in the hot path.

Why most teams get this wrong

Teams tend to fall off one of two sides, and both are expensive in different ways.

One group overbuilds. They ship a routing layer more elaborate than the feature it serves, complete with a classifier and a config system, and now they maintain a small product just to pick a model. The other group ignores routing completely and quietly pays frontier prices for every trivial call. The boring middle, a few static rules by task and plan plus a spend triggered downgrade, beats both.

The simplest routing that pays for itself

If you only ever do one piece of routing, make it the spend triggered downgrade, because it protects your worst case with zero ongoing rules to tune. That is what Weckr gives you built in.

import { Weckr } from '@weckr/sdk'

const wk = new Weckr({
  apiKey: process.env.WECKR_API_KEY,
  plans: { free: 0, pro: 29 },
})

// You still route by task or plan in your own code if you want. Weckr adds the
// spend based rule automatically: over cap, this request goes to the cheaper model.
const response = await wk.chat(openai, {
  model: 'gpt-4o',
  messages,
  userId: user.id,
  feature: 'chat',
  plan: user.plan,
})

You keep your task and plan rules where they belong, in your code, and let Weckr handle the one routing rule that is annoying to build well: downgrade on spend. Set it up in the docs, see it on seeded data at the demo, and fit it into the bigger picture in the guide to AI cost and margin management.

FAQ

Route the cheap work to a cheap model

Most of your requests do not need your best model, and paying as if they do is the quietest way to wreck an AI margin. A few routing rules fix it, and the spend based one is the highest value and the most tedious to build.

Weckr ships that rule for you, plus cost and margin per user so you can see which features to route down next. Start at the demo.