Failure Modes

The “Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey” (2026) paper discusses a few failure modes:

• Distribution shift
• Router staleness
• Cost of router errors
• Routing collapse