Case study

LLMRouter

“LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers” (2026), arXiv:2608.06867

LLMRouter is interesting because it combines three different goals

Cost efficiency
Capability matching
User preference

...in a unified framework

Single-turn
Multi-turn
Personalized

Diagram of LLMRouter's unified formulation: three motivating scenarios (cost efficiency, capability matching, user preference) feed into a routing state (query, persona, history) shared across single-turn, multi-turn, and personalized routing, which is scored by a router combining a context encoder and model encoder over LLM candidates. — LLMRouter's unified formulation across single-turn, multi-turn, and personalized routing

LLMRouter treats routing as full infrastructure, not just a scoring function: a Data Engine builds training data, a Router Trainer fits the routing model, a Route Engine scores and dispatches each request at inference time, and separate Evaluation and Deployment layers plug into existing serving stacks.

Architecture diagram of LLMRouter with six modules: Data Engine, Router Library, Router Trainer, Route Engine, Evaluation, and Deployment, connected in a pipeline from data curation through training to inference and deployment. — Figure 3: Architecture of LLMRouter — six modules covering data, training, inference, evaluation, and deployment