Model routing
Sending each request to the cheapest model that can handle it, and only the hard cases to the expensive one.
01In short
Sending each request to the cheapest model that can handle it, and only the hard cases to the expensive one.
02Video
slot · videoModel routing in 90 secondsadd: GUIDES['model-routing'].video
03Guide
slot · guide
A step-by-step guide for “Model routing” goes here. Suggested outline:
- What it is — in one paragraph
- Why it matters in production
- How to do it — 3 to 7 steps
- Pitfalls we see in the field
04Checklist
slot · checklist
Four to eight things a team can tick before go-live.
05FAQ
slot · FAQ
The three questions clients actually ask about “Model routing”.
06Related terms
BasicsInferenceRunning a trained model to get an output. In production it is the part you pay for on every single request.BasicsReasoning modelA model trained to work through a problem in intermediate steps before answering. Stronger at maths, code and multi-step tasks; slower and pricier per answer.
Where we help · Enable