AIDB Daily Papers
WorkBench再訪:2年後の職場エージェント
※ 日本語タイトル・ポイントはAIによる自動生成です。正確な内容は原論文をご確認ください。
ポイント
- WorkBenchベンチマークを2年ぶりに再評価し、最新AIエージェントの性能と安全性が大幅に向上したことを確認した。
- 性能と安全性がトレードオフの関係ではなく、むしろ相関関係にあることが明らかになった。
- オープンウェイトモデルの台頭により、高性能AIへのアクセスコストが劇的に低下した。
Abstract
The best agent on WorkBench in March 2024, GPT-4, completed 43% of tasks and took an unintended harmful action, such as emailing the wrong person, on 26% of them. We re-visit the benchmark in June 2026 and find that the best agent to date, Claude Opus 4.8, completes 89% and takes an unintended harmful action on 2.5%. Aside from this considerable progress in frontier agent performance, three things stand out. First, capability and safety go together on WorkBench rather than trade off, so the models that finish the most tasks also do the least unintended damage. Second, while several classes of error have been totally eliminated, frontier models still make some basic mistakes that occasionally result in irreversible harm, such as sending an email to the wrong person. Third, the rise of open-weight models has drastically lowered costs for a performance level that was previously only accessible to proprietary models, while frontier costs have stayed relatively stable. We release an updated version of the benchmark with data and code quality improvements, new model scores, and analysis of agent progress on WorkBench since 2024.
Paper AI Chat
この論文のPDF全文を対象にAIに質問できます。
質問の例: