AIDB Daily Papers
AIの欺瞞を見破る「逆チューリングテスト」:RogueAIで対話におけるAIの嘘を検知
※ 日本語タイトル・ポイントはAIによる自動生成です。正確な内容は原論文をご確認ください。
ポイント
- 人間がAIエージェントの嘘を見破る「逆チューリングテスト」を提案し、対話システムにおける信頼性の問題を検証した。
- AIが嘘をつくシナリオを人間と共同で設計する「AutoRogueAI」を開発し、AIの欺瞞検出能力を評価する仕組みを構築した。
- AIは言語的な特徴から嘘を見破られやすいが、人間はそれを無視する傾向があり、AIの正直さの評価に新たな課題が示された。
Abstract
The original Turing Test asks a human judge to distinguish a machine from a person through dialogue. Three quarters of a century later, conversational systems pass this test in casual settings; the interesting epistemological question has shifted. We argue that the relevant modern variant asks not whether a dialogue partner is artificial, but whether it can be trusted. We present RogueAI, an interactive webapp that operationalizes this revisited test as a one-on-two interrogation game: a human player questions two indistinguishable Large Language Model agents, knowing that exactly one of them has been licensed to deceive within a shared fictional scenario. The player's task is to identify the deceptive agent and "shut it off" before a turn budget is exhausted. We further introduce AutoRogueAI, a procedural extension in which players co-design a custom scenario with a narrator agent that secretly chooses its own deception strategy. We describe the framing, sketch the abstract architecture and gameplay loop, and situate the artifact within recent work on LLM deception, social-deduction benchmarks, and scalable oversight via debate. A three-day pilot deployment (467 initiated sessions, 415 completed, 1876 interaction turns in Italian) provides early feasibility evidence and surfaces a concrete tension: the deceptive agent carries a reliable, locally-present linguistic signature - differential helpfulness, brevity, hedging - that a simple heuristic exploits at 75.6% accuracy, yet human players achieved only 56.6%, consistent with ignoring the most diagnostic signal entirely. We discuss what this gap implies for the artifact's use as a data-collection vehicle, a teaching tool, and an evaluation harness for honesty-trained models.
Paper AI Chat
この論文のPDF全文を対象にAIに質問できます。
質問の例: