RealCLI: a paired benchmark for unified evaluation of task execution and human–agent collaboration on realistic CLI tasks
Read the original at arxiv.org→arXiv:2610.22507v1 Announce Type: new Abstract: Command-line interface (CLI) agents increasingly execute longer-horizon tasks, yet existing benchmarks either assume that user instructions remain fixed throughout...
Original headline: "RealCLI: Toward a Unified Evaluation of Human-Agent Collaboration for CLI Agents"
Coverage timeline
- Oct 19, 04:00 UTC arXiv cs.CL lead source RealCLI: Toward a Unified Evaluation of Human-Agent Collaboration for CLI Agents