AIDB Daily Papers
AIネイティブ開発チームのためのリスクアーキテクチャ:自律型システムガバナンスの組織的フレームワーク
※ 日本語タイトル・ポイントはAIによる自動生成です。正確な内容は原論文をご確認ください。
ポイント
- 自律型AIシステムを開発・運用するチーム向けに、役割や意思決定権、エスカレーション構造を定義する新しい組織的リスク管理フレームワークを提案した。
- 従来のソフトウェア工学の前提が崩れるAIネイティブな環境において、チームの特性を分類し、特有の失敗モードを体系化した点が新しい。
- AIネイティブな運用へ移行するにつれリスク管理の網羅性が低下し、特に決定論的なシステムとの境界で深刻な失敗が発生しやすいことを明らかにした。
Abstract
Engineering management research has produced mature frameworks for software risk: ownership by feature, escalation by severity, and assurance by test coverage. These frameworks implicitly assume deterministic behavior, discrete and auditable change events, and clear component-to-owner mappings. Teams that build and operate agentic AI systems violate all three assumptions at once: outputs are probabilistic, systems take autonomous multi-step actions, and the risk surface mutates silently between deployments. Existing AI risk literature addresses this from above (policy frameworks such as the NIST AI RMF and ISO/IEC 42001) or below (threat taxonomies such as OWASP's agentic AI guidance), but not at the layer where an engineering manager (EM) operates: roles, decision rights, and escalation structures. This paper contributes (i) a seven-dimension profile distinguishing pure software-engineering, hybrid, and AI-native teams; (ii) a six-cluster failure-mode taxonomy including a previously unarticulated cluster, dependency-boundary determinism mismatch; and (iii) a synthetic framework-adequacy methodology scoring how well each profile's risk architecture detects, contains, and escalates a defined scenario set. Because the object of study is framework adequacy rather than human behavior, the evaluation yields derived rather than observed coverage claims. Coverage degrades as teams move from pure software engineering to AI-native operation, monotonically in the median and abruptly in the count of uncovered, high-consequence failures appearing only at the AI-native step. The degradation concentrates in specific failure-mode categories, and the most severe, least-covered failures arise not inside AI-native teams but at the organizational boundary where their probabilistic outputs are consumed by determinism-assuming dependencies.
Paper AI Chat
この論文のPDF全文を対象にAIに質問できます。
質問の例: