- Several benchmarks score every prompt on every model in the pool, then report the oracle ceiling — the accuracy you would reach if you always picked the right model: RouterBench (Hu et al., 2024), LLMRouterBench (Li et al., 2026), RouterArena (Lu et al., ICLR 2026)
- “The Routing Plateau: Understanding and Breaking the Accuracy Limits of LLM Routers” (Lu et al., 2026) — 21 routing methods across five benchmarks
- On RouterBench the top 15 routers differ by 0.23 percentage points — diverse designs, one shared ceiling
- Plain kNN ranks in the top 2 on all five benchmarks
- Hard queries are 11–35% of each benchmark but account for 70–91% of the oracle gap
- Best remedy so far: 10× more training data, a stronger encoder, and end-to-end fine-tuning together add 2.13 pp and close 14.6% of the gap
- LLMRouterBench (400K+ instances, 21 datasets, 33 models): the best routers land a few points above the best single model, while the oracle sits roughly 20 points above the best router