AIDB Daily Papers
ルーティングはいつ有意義になるのか?言語モデル社会における多様性と堅牢性
※ 日本語タイトル・ポイントはAIによる自動生成です。正確な内容は原論文をご確認ください。
ポイント
- 言語モデルのルーティング手法において、精度やコスト以外の評価指標として行動の多様性と安定性を定義した。
- 階層的社会エントロピーと摂動ベースの堅牢性指標を導入し、モデル社会の質を診断する手法を提案した。
- 少数のモデルで多様性が十分に確保できることや、精度とルーティングの有意義さが必ずしも一致しないことを示した。
Abstract
Routing policies for multi-model systems are evaluated almost exclusively on task accuracy and inference cost. We argue that two properties, orthogonal to performance, determine whether routing is meaningful. First, the society of actors must be behaviourally differentiated: if all actors respond identically, routing is vacuous. Second, the routing policy must be stable: surface-form variants of a query should be assigned to the same actor. High task accuracy is compatible with violating both properties, since a router can operate over a redundant society or assign queries inconsistently, preventing specialisation regardless of performance. We adapt Hierarchic Social Entropy (HSE) to language-model societies and introduce a perturbation-based robustness metric to diagnose these failure modes. Applied to EmbedLLM and RouterBench, we find that HSE exhibits strong diminishing returns, suggesting that a curated subset of fewer than ten agents recovers most available diversity in a large pool -- a practical coreset heuristic for society design. We further find that KNN routers gain accuracy from specialist societies but collapse in robustness under perturbation, while prompted routing remains stable across all perturbation types -- illustrating that accuracy and meaningfulness can sharply diverge.
Paper AI Chat
この論文のPDF全文を対象にAIに質問できます。
質問の例: