AIDB Daily Papers
推論の変容:コンテキストはいかにLLMの推論を静かに短縮するか
※ 日本語タイトル・ポイントはAIによる自動生成です。正確な内容は原論文をご確認ください。
ポイント
- LLMの推論における頑健性を評価するため、様々なコンテキスト下でモデルの挙動を分析しました。
- 問題解決の文脈が変化すると、LLMの推論過程が短縮され、自己検証などの行動が減少することがわかりました。
- この変化は単純な問題では影響が少ないものの、複雑なタスクでは性能低下につながる可能性があります。
Abstract
Large language models (LLMs) exhibiting test-time scaling behavior, such as extended reasoning traces and self-verification, have demonstrated remarkable performance on complex, long-term reasoning tasks. However, the robustness of these reasoning behaviors remains underexplored. To investigate this, we conduct a systematic evaluation of multiple reasoning models across three scenarios: (1) problems augmented with lengthy, irrelevant context; (2) multi-turn conversational settings with independent tasks; and (3) problems presented as a subtask within a complex task. We observe an interesting phenomenon: reasoning models tend to produce much shorter reasoning traces (up to 50%) for the same problem under different context conditions compared to the traces produced when the problem is presented in isolation. A finer-grained analysis reveals that this compression is associated with a decrease in self-verification and uncertainty management behaviors, such as double-checking. While this behavioral shift does not compromise performance on straightforward problems, it might affect performance on more challenging tasks. We hope our findings draw additional attention to both the robustness of reasoning models and the problem of context management for LLMs and LLM-based agents.
Paper AI Chat
この論文のPDF全文を対象にAIに質問できます。
質問の例: