CogArena：在交互式行为实验上评测多模态智能体
CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments
以往对大语言模型认知能力的检验多把行为范式改写成文字提示，绕开了人类被试实际面对的视觉界面和时间压力。作者构建 CogArena，收录十个社会与行为科学实验范式，智能体通过标准网页浏览器操作与在线被试相同的视觉刺激和反应界面。评分分三级，分别考察是否完成任务、完成得是否准确、数据是否呈现该范式在人群中稳定出现的行为特征。四个多模态模型在每个任务上各运行十次，结果显示完成实验与复现行为特征并不等同，不少会话未能完成。作者公开了任务、评分代码和排行榜。
发布十个社会与行为科学实验范式的浏览器端基准，四个多模态模型各跑十次，按完成、准确率和行为特征三级评分。
arXiv:2610.36484v1 Announce Type: cross Abstract: Claims about the cognitive capabilities of large language models are typically tested by translating behavioral paradigms into text prompts, bypassing the visual interfaces and time pressure through which human participants actually engage with experiments. At the same time, AI agents can now complete online behavioral tasks and produce human-like data, which raises concerns that online datasets in the social and behavioral sciences are increasingly open to contamination. We introduce CogArena, a benchmark of ten experimental paradigms from the social and behavioral sciences. Agents complete the experiments through a standard web browser, working with the same visual stimuli and response interfaces that online participants use. A three-level scoring procedure evaluates whether the agent completed the task, how accurately it performed, and whether its data show the behavioral signatures each paradigm reliably produces in people. Across four multimodal models, each run ten times per task, we find that finishing an experiment and reproducing its signatures are different things. Many sessions did not finish, and those that did reproduced some signatures reliably and others not at all. CogArena provides the tasks, scoring code, and a public leaderboard for new submissions.
中文翻译还没有生成，先显示原文。
来源：arXiv · Human-Computer Interaction · arxiv.org