“RouteLLM: Learning to Route LLMs with Preference Data” — Ong et al. (2024), arXiv:2406.18665 · LMSYS blog
- Learns a routing policy directly from human preference data (Chatbot Arena), instead of hand-tuned rules
- Trained four router architectures — similarity-weighted ranking, matrix factorization, a BERT classifier, and a causal-LLM classifier — and open-sourced all of them
- Cut cost by over 85% on MT-Bench, 45% on MMLU, and 35% on GSM8K versus always calling GPT-4, while keeping about 95% of GPT-4’s quality
- One of the first routers released with reusable benchmarks and trained models — it became a common baseline for the work that followed