[Paper] CollabSim: A CSCW-Grounded Methodology for Investigating Collaborative Competence of LLM Agents through Controlled Multi-Agent Experiments
Source: arXiv - 2606.06399v1
Overview
Multi-agent systems (MAS) built on large language models have shown growing promise, with their effectiveness resting on agents’ ability to coordinate through text-based channels much as human teams do. Yet recent study suggests that MAS often falter not because agents lack individual task-solving ability, but because they lack collaborative competence: the capacity to establish common ground, maintain shared task understanding, balance individual and collective incentives, and repair misalignment as interaction unfolds. Decades of research in Computer-Supported Cooperative Work have characterized these requirements for human teams coordinating under constrained communication, yet existing MAS evaluations focus mainly on task outcomes or single-agent proficiency in reasoning, planning, and tool use. To enable a systematic analysis of agents’ collaborative competence in MAS, we introduce CollabSim, a configurable simulation framework that combines a theory-grounded definition of collaborative capabilities, controlled manipulation of interaction conditions, and action-level probing of agents’ internal states. Experiments across four LLMs show that CollabSim can capture condition effects, separate model performance patterns, and reveal task-dependent effects of agent design.
Key Contributions
This paper presents research in the following areas:
- cs.CL
Methodology
Please refer to the full paper for detailed methodology.
Practical Implications
This research contributes to the advancement of cs.CL.
Authors
- Jiaju Chen
- Bo Sun
- Yuxuan Lu
- Yun Wang
- Dakuo Wang
- Bingsheng Yao
Paper Information
- arXiv ID: 2606.06399v1
- Categories: cs.CL
- Published: June 4, 2026
- PDF: Download PDF