今日速览
| 序号 | 标题 | 来源 | 日期 | 主题 | 推荐等级 |
|---|---|---|---|---|---|
| 1 | PACE: A Proxy for Agentic Capability Evaluation | arXiv | 2026-07-02 | AI-Agent | 高 |
| 2 | A-TMA: Decoupling State-Aware Memory Failures in Long-Term Agent Memory | arXiv | 2026-07-02 | AI-Agent | 高 |
重点论文与技术动态
1. PACE: A Proxy for Agentic Capability Evaluation
- 来源：arXiv
- 日期：2026-07-02
- 作者/机构：Yueqi Song, Lintang Sutawika, Jiarui Liu, Lindia Tjuatja, Jiayi Geng, Yunze Xiao
- 主题标签：
AI-Agent,arXiv - 推荐等级：高
- 分类：cs.AI, cs.CL
一句话结论
PACE框架通过小规模代理基准可靠预测大型代理评估性能，实验表明其成本仅为全评估的1%以下，预测精度高（MAE<4%，Spearman相关>0.80），显著降低开发与评估开销。
核心内容
- 代理评估基准（如SWE-Bench、GAIA）成本高昂（单次数千美元、需数天），需复杂基础设施，而原子能力评估（如推理、代码生成）快速经济。
- PACE从非代理评估中精选关键实例子集，通过回归模型映射模型在源实例的分数到目标代理基准分数。
- 实验在14个模型、4个代理基准和19个非代理基准上验证，预测MAE低于4%，Spearman相关高于0.80，模型排名准确率约85%。
方法与数据
- PACE结合目标相关局部选择和全局信息全局选择策略优化实例子集。
- 基于19个非代理基准的实例数据构建PACE-Bench，应用于14个模型与4个代理基准的实验。
价值判断
- 值得关注：将代理评估成本降至全评估的1%以下，使模型开发、选择和路由中的实时性能估计成为可能。
- 可复用点：框架可直接迁移至其他代理评估任务，无需重新设计基础设施。
- 局限/待核查：摘要未明确提及具体局限性。
摘要
Evaluating LLM agents on benchmarks like SWE-Bench and GAIA can be expensive, time-consuming, and requires complex infrastructure. A single evaluation can cost thousands of dollars and take days to complete. In contrast, non-agentic LLM benchmarks that test individual capabilities (e.g., reasoning, code generation) are fast and cheap to run. In this paper, we investigate whether performance on expensive agentic benchmarks can be accurately predicted by the performance on a small, carefully selected subset of atomic evaluation instances. We introduce PACE, a framework that constructs proxy benchmarks by selecting instances from existing non-agentic evaluations whose aggregate scores most reliably predict model performances on agentic benchmarks. Given a pool of candidate instances spanning atomic capabilities, PACE fits a regression that maps a model's scores on a compact subset of source instances to its score on the target agentic benchmark. The subset itself is curated by combining two complementary instance-selection strategies, target-relevance local selection and globally informative global selection. We apply PACE to the 4 target agentic benchmarks in this paper, which yields PACE-Bench, the concrete proxy benchmark that we evaluate in the paper. Experiments across 14 models, 4 agentic benchmarks, and 19 non-agentic benchmarks show that PACE-Bench predicts agentic scores with leave-one-out cross-validation (LOOCV) mean absolute error (MAE) under 4%, Spearman correlation above 0.80, and pairwise model-ranking accuracy around 85%, all at much less than 1% of the full agentic evaluation cost. We further analyze the selected proxy instances, revealing which skills each agentic benchmark uniquely demands. PACE enables practitioners to obtain reliable estimates of agentic performance during model development, selection, and routing, without the overhead of full agent evaluation.2. A-TMA: Decoupling State-Aware Memory Failures in Long-Term Agent Memory
- 来源：arXiv
- 日期：2026-07-02
- 作者/机构：Zitong Shi, Yixuan Tang, Anthony Kum Hoe Tung
- 主题标签：
AI-Agent,arXiv - 推荐等级：高
- 分类：cs.AI
一句话结论
ATMA通过显式状态角色管理，有效缓解长期代理记忆中的幽灵记忆问题，提升系统准确性。
核心内容
- 幽灵记忆指旧事实、当前事实和过渡事实在记忆库中混合，导致检索时误导答案模型。
- ATMA作为状态感知覆盖层，保留过时记录，构建查询所需状态视图的证据包，并暴露当前、历史和过渡标签。
- 提出LTP基准评估幽灵记忆，在LTP上冲突准确率提升0.240，在LoCoMo上时间F1从0.0295升至0.1705。
方法与数据
- 采用LTP（LoCoMo Temporal Plus）冲突密集基准和LoCoMo进行评估。
- 摘要未明确具体数据集细节。
价值判断
- 值得关注：幽灵记忆问题常被最终QA准确率掩盖，需针对性解决长期记忆可靠性。
- 可复用点：ATMA框架可直接集成到现有记忆系统，优化银行维护、检索和答案解析三层。
- 局限/待核查：性能提升依赖主机环境，需跨平台验证泛化性。