- 2023
- FrugalGPT: first influential paper framing routing as a cost problem, using cascades of cheap-to-expensive models
- 2024
- RouteLLM: learns to route from human preference data, open-sourced trained routers and reusable benchmarks
- 2025
- Wider adoption: routing shows up in production systems (vLLM Semantic Router launches); research diversifies into RL, uncertainty, and personalization-based routers
- 2026
- Maturing and sobering up: unified infrastructure (LLMRouter), standardized benchmarks (LLMRouterBench), and evidence that today’s routers plateau well below what’s theoretically possible (The Routing Plateau)
The rest of this talk follows that arc: what routing tries to optimize, how it’s built, and how far it still has to go.