AIDB Daily Papers
言語を超えたAIの推論基盤:形式に依存しない「FARS」を発見
※ 日本語タイトル・ポイントはAIによる自動生成です。正確な内容は原論文をご確認ください。
ポイント
- 大規模言語モデルが持つ、言語・コード・数式など形式に依存しない共通の推論基盤「FARS」を特定しました。
- この研究は、AIの推論メカニズムを形式を超えて理解する新たな視点を提供し、AIの汎用性を探る上で重要です。
- FARSはモデルの中間層に存在し、特定の10次元で概念構造を増幅させ、形式情報を抑制することで、高い汎用性を示しました。
Abstract
Large language models represent the same reasoning in vastly different surface forms -- English prose, Python code, mathematical notation -- yet whether they share a common internal substrate across these symbolic systems remains unknown. We introduce the TriForm Benchmark (18 concepts x 6 forms x 3 instances = 324 stimuli) and study five LLMs (1.6B-8B) across three architecture families. Using permutation-corrected RSA, cross-form probing, and activation patching, we find converging evidence for a Format-Agnostic Reasoning Subspace (FARS) in middle layers. We make FARS concrete: concept-centroid PCA extracts a 10-dimensional subspace that amplifies concept structure 3x while suppressing form information to near zero. Replacing only these 10 dimensions during cross-form patching preserves 90-96% of model output -- far exceeding both full activation replacement (44-56%) and variance-maximizing PCA (60-74%) -- while ablating them causes targeted disruption. FARS generalizes to held-out concepts and converges across architectures (CCA > 0.79 for all model pairs), providing within-modality evidence for the Platonic Representation Hypothesis. We further discover a declarative-procedural asymmetry: representations are far more compatible between prose and mathematics than between either and code, suggesting that the critical axis of divergence is not linguistic vs. formal but declarative vs. procedural.
Paper AI Chat
この論文のPDF全文を対象にAIに質問できます。
質問の例: