AIDB Daily Papers
LLM管理マルチエージェントシステムにおける反復監査収束:プロンプトエンジニアリング品質保証の事例研究
※ 日本語タイトル・ポイントはAIによる自動生成です。正確な内容は原論文をご確認ください。
ポイント
- 本研究では、7150行のプロンプト仕様を持つ実稼働システムAEGISに対し、Claudeエージェントによる反復的な監査を実施した。
- この研究は、大規模言語モデル(LLM)システムにおけるプロンプト仕様の品質保証手法として、エージェント主導の監査の有効性を示す点で重要である。
- 9回の監査ラウンドで51件の一貫性欠陥が発見され、非単調な収束と監査プロトコルが明らかになった。
Abstract
Prompt specifications for multi-agent large language model (LLM) systems carry data contracts and integration logic across many interdependent files but are rarely subjected to structured-inspection rigor. This paper reports a single-system empirical case study of iterative, agent-driven auditing applied to AEGIS (Autonomous Engineering Governance and Intelligence System), a production seven-lane orchestration pipeline whose prompt-specification surface comprises approximately 7150 lines: 6907 across seven lane PROMPT.md files and a 245-line shared Ticket Contract. Nine sequential audit rounds, executed by Claude sub-agents using a checklist-driven walkthrough adapted from Weinberg and Freedman, surfaced 51 prompt-specification consistency defects, distinct from the 51 STRIDE-categorized adversarial code findings reported in the companion preprint. Per-round counts were 15, 8, 12, 2, 8, 1, 4, 1, and 0. We report a seven-category post-hoc defect taxonomy with explicit coding rules, observed non-monotonic convergence consistent with cascading edits and audit-scope expansion, and an audit protocol distilled from the study, with the final locked checklist released as a reproducibility appendix. Single-file review missed defect classes that were surfaced only by later expanded-scope rounds in this system. The same LLM family authored and audited the specifications; replication with dissimilar models and human reviewers is required before generalization.
Paper AI Chat
この論文のPDF全文を対象にAIに質問できます。
質問の例: