Guide

Choosing which model to route to

Aniron routes to 20+ open-source and frontier models behind one API. The right model for a request is whichever one clears your quality bar at the lowest cost and latency — not whatever is newest or largest. This guide gives a framework for that trade-off, plus how to encode it in a request.

Three questions to ask per task

How costly is a wrong answer?

A misclassified support ticket is cheap to fix. A hallucinated number in a customer-facing summary or a broken code generation is not. High-stakes, low-volume steps justify a frontier model's cost. High-volume, low-stakes steps do not.

What's the latency budget?

An autocomplete suggestion needs a response in a few hundred milliseconds. An overnight batch summarization job can tolerate multi-second latency per call in exchange for a stronger model. Match the tier to the interaction, not the other way around.

What's the actual request volume?

At a thousand requests a month, the cost gap between tiers is noise. At a million a day, routing everything through a frontier model by default can be the difference between a sustainable margin and none. Recompute the trade-off as volume grows.

Model tiers at a glance

Tier	Example models	Cost	Latency	Best for
Frontier	aniron/gpt-4o, aniron/claude-sonnet-4	$2.50–$10 / M tokens	Higher (1–4s first token)	Multi-step reasoning, code generation, ambiguous instructions, anything customer-facing where a wrong answer is costly.
Mid-tier open	aniron/llama-3.3-70b, aniron/mixtral-8x22b	$0.35–$0.90 / M tokens	Moderate (0.5–1.5s first token)	Summarization, classification, extraction, RAG answer synthesis, internal tools with a human in the loop.
Small / fast	aniron/llama-3.1-8b, aniron/qwen-2.5-7b	$0.05–$0.15 / M tokens	Low (under 500ms first token)	Routing/intent detection, autocomplete, tagging, high-volume batch jobs, anything latency-sensitive at scale.

Exact rates and the full model list are on /pricing and /models. Rates change as new models are added — treat this table as directional.

Encoding the decision in a request

Pick a model per request

The model parameter is just a string, so branching on your own task metadata — stakes, volume, latency budget — to pick a tier is a normal conditional in your application code.

// Cost-and-latency-aware routing per request
const completion = await client.chat.completions.create({
  model: task.isHighStakes
    ? 'aniron/claude-sonnet-4'
    : 'aniron/llama-3.1-8b',
  fallbacks: ['aniron/llama-3.3-70b'],
  messages,
});

Chain a fallback for resilience

Route to a cheap model by default and let the fallbacks array escalate to a stronger one only when the primary model errors or rate-limits, instead of paying frontier cost on every call.

const completion = await client.chat.completions.create({
  model: 'aniron/llama-3.1-70b',
  fallbacks: [
    'aniron/llama-3.3-70b',
    'aniron/claude-sonnet-4'
  ],
  messages: [{ role: 'user', content: 'Explain TCP.' }],
});

// Cheap model first, escalate to a stronger one only on
// rate limit, service error, or timeout — not on every call.

Frequently asked questions

Should I default to the most capable model everywhere?

No. Frontier models cost 10-50x more per token than small models and usually add latency. Reserve them for steps where a mistake is expensive — final answers, code that runs in production, anything user-facing. Route everything else to a cheaper tier and measure quality before upgrading.

How do I decide between a mid-tier open model and a frontier model?

Run the same prompt set against both and compare outputs on your actual task, not a generic benchmark. If the mid-tier model matches quality on your data, use it. Reasoning-heavy or multi-turn agentic tasks tend to need the frontier tier; single-shot extraction and summarization usually do not.

What does the fallbacks array actually do?

It lists backup models to retry with, in order, if the primary model returns a 429 rate limit, a 503, or times out after 60 seconds. You are only billed for the model that completes the request — failed attempts before a fallback do not consume credits.

Can I route different steps of one pipeline to different models?

Yes. Each request carries its own model parameter, so a multi-step pipeline (retrieve, then summarize, then answer) can call a small model for retrieval-adjacent steps and a frontier model only for the final synthesis step.

Does switching models require touching my prompts?

Usually not for the request shape, since every model is called with the same OpenAI-compatible messages array. Prompt wording that is tuned for one model family sometimes needs light adjustment when you move to a very different one — test before shipping the switch.

How do I estimate cost before committing to a model for a workload?

Take a representative sample of real requests, run them once against each candidate model, and multiply the observed average tokens per request by that model's rate on /pricing. That gives a per-request cost you can scale to expected volume.

Related resources