次回の更新記事:誤解を招きやすいAI用語6選、技術語なのに揺れる意味(公開予定日:2026年04月30日)
AIDB Daily Papers

爪と危険:オープンエージェントシステムは信頼できるか?

原題: Clawed and Dangerous: Can We Trust Open Agentic Systems?
著者: Shiping Chen, Qin Wang, Guangsheng Yu, Xu Wang, Liming Zhu
公開日: 2026-03-27 | 分野: LLM セキュリティ 機械学習 AI ソフトウェア エージェント リスク 評価 倫理 制御 システム 設計 自然言語処理 論文

※ 日本語タイトル・ポイントはAIによる自動生成です。正確な内容は原論文をご確認ください。

ポイント

  • LLM 기반 계획, 외부 기능, 지속적인 메모리, 권한 있는 실행을 결합한 오픈 에이전트 시스템의 보안 문제를 분석했다.
  • 기존 소프트웨어와 달리 예측 불가능한 실행과 불확실한 환경으로 인해 새로운 보안 문제가 발생하며, 에이전트 행동 통제가 중요하다.
  • 공격, 벤치마크, 방어 등 50편의 논문을 분석하여 보안 설계된 에이전트 플랫폼을 위한 참조 지침과 평가 점수표를 제시했다.

Abstract

Open agentic systems combine LLM-based planning with external capabilities, persistent memory, and privileged execution. They are used in coding assistants, browser copilots, and enterprise automation. OpenClaw is a visible instance of this broader class. Without much attention yet, their security challenge is fundamentally different from that of traditional software that relies on predictable execution and well-defined control flow. In open agentic systems, everything is ''probabilistic'': plans are generated at runtime, key decisions may be shaped by untrusted natural-language inputs and tool outputs, execution unfolds in uncertain environments, and actions are taken under authority delegated by human users. The central challenge is therefore not merely robustness against individual attacks, but the governance of agentic behavior under persistent uncertainty. This paper systematizes the area through a software engineering lens. We introduce a six-dimensional analytical taxonomy and synthesize 50 papers spanning attacks, benchmarks, defenses, audits, and adjacent engineering foundations. From this synthesis, we derive a reference doctrine for secure-by-construction agent platforms, together with an evaluation scorecard for assessing platform security posture. Our review shows that the literature is relatively mature in attack characterization and benchmark construction, but remains weak in deployment controls, operational governance, persistent-memory integrity, and capability revocation. These gaps define a concrete engineering agenda for building agent ecosystems that are governable, auditable, and resilient under compromise.

Paper AI Chat

この論文のPDF全文を対象にAIに質問できます。

質問の例:

AIチャット機能を利用するには、ログインまたは会員登録(無料)が必要です。

会員登録 / ログイン

💬 ディスカッション

ディスカッションに参加するにはログインが必要です。

ログイン / アカウント作成 →

関連するAIDB記事