今日速览
| 序号 | 标题 | 来源 | 日期 | 主题 | 推荐等级 |
|---|---|---|---|---|---|
| 1 | AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement | arXiv | 2026-08-20 | RAG | 高 |
| 2 | Break It Down, Pass It On: Cross-Task Skill Transfer in LLM Agents | arXiv | 2026-08-20 | RAG | 高 |
重点论文与技术动态
1. AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement
- 来源：arXiv
- 日期：2026-08-20
- 作者/机构：Yizhe Chi, Wenyi Li, Deyao Hong, Xiaoqiu Wang, Mingju Gao, Kaisen Yang
- 主题标签：
RAG,arXiv - 推荐等级：高
- 分类：cs.AI, cs.CL, cs.LG
一句话结论
AI4AI-Bench基准测试表明LLM代理在递归自我改进中仅能微弱提升训练算法性能，最佳系统得分0.250，仅接近原算法与最优距离的25%。
该研究构建AI4AI-Bench评估LLM代理设计训练算法的能力，结果显示平均得分0.166，最佳系统0.250，多数代理未改变模型学习方式，少数改变的平均得分0.226显著高于其余0.126，但整体改进空间巨大。
核心内容
- 提出AI4AI-Bench基准，包含10个冻结研究仓库覆盖10个训练算法家族，每个任务映射到统一尺度（0为无信息，0.1为原算法，1.0为最优）。
- 代理在4小时内重写训练算法，代码重跑12小时评分，10个指标不相容但统一量化。
- 多数代理不改变模型学习方式（仅8%尝试），少数改变的平均得分0.226，显著高于其余0.126。
方法与数据
- 10个任务基于冻结仓库，代理有4小时在B300上重写算法，代码重跑评分。
- 摘要未明确具体数据集细节。
价值判断
- 值得关注：递归自我改进可行性验证，填补了现有基准空白。
- 可复用点：释放任务套件、评估器及所有评分提交，支持可复现性研究。
- 局限/待核查：平均得分仅0.166，改进空间大；代理大多不尝试改变学习方式。
摘要
Recursive self-improvement (RSI) asks whether an AI system can improve the process that produces AI systems, so that the next system inherits the improvement. That process is the training algorithm: a better objective or update rule improves the compute\mbox{-}capability exchange rate for every subsequent run, including the one that produces the next agent. Whether RSI is feasible therefore turns on whether an agent can design training algorithms. No benchmark isolates that ability: existing suites are won by collecting data or by tuning hyperparameters, and none tells a change to how a run is executed apart from a change to how the model learns. We present AI4AI\mbox{-}Bench, 10 frozen research repositories spanning 10 training algorithm families. In each task, an agent has 4 hours on one B300 to rewrite the training algorithm; its code is then rerun from scratch for up to 12 hours and scored by a fixed evaluator hidden from the agent, against the repository's original algorithm under the same procedure. Because the 10 metrics are incommensurable, every task is mapped onto one scale on which $0$ is an uninformative model, $0.1$ is the algorithm the repository ships, and $1.0$ is the task optimum. Across 29 configurations of 6 systems on all 10 tasks the mean score is $0.166$, and the best system reaches $0.250$: even the strongest closes under a fifth of the distance between the algorithm that was already there and the optimum. The submissions show where that distance went: most never change how the model learns at all, and the minority that do average $0.226$ against $0.126$ for the rest. More reasoning effort mostly buys the willingness to go there, taking that minority from $8\%$ of submissions to $64\%$ and the mean score from $0.094$ to $0.196$. We release the task suite, the evaluators and every scored submission, so that the measurement can be repeated as these systems change.2. Break It Down, Pass It On: Cross-Task Skill Transfer in LLM Agents
- 来源：arXiv
- 日期：2026-08-20
- 作者/机构：Yiyang Feng, Biddut Sarker Bijoy, Niranjan Balasubramanian, Jiawei Zhou
- 主题标签：
RAG,arXiv - 推荐等级：高
- 分类：cs.AI, cs.CL
一句话结论
子任务级技能和文本格式技能能显著提升LLM代理在跨任务技能转移中的可靠性。
一段话：本研究通过系统比较任务级与子任务级技能诱导及文本与代码技能格式，发现子任务级技能平均提升代理性能（高于无记忆基线），任务级技能则降低性能；文本技能转移效果优于代码技能。进而提出技能效用分数，结合具体性与抽象性，能一致预测技能转移成功率，且计算仅需技能和任务描述，无需任务执行。
核心内容
- 子任务级技能平均提升代理性能，任务级技能则降低性能。
- 文本技能在跨任务转移中表现优于代码技能。
- 技能效用分数通过结合具体性与抽象性，能一致预测技能转移成功率。
方法与数据
- 摘要未明确具体实验数据。
价值判断
- 值得关注：技能效用分数无需任务执行即可实时诊断技能质量，提升代理效率。
- 可复用点：计算仅需技能和任务描述，方法轻量易集成至现有LLM代理系统。
- 局限/待核查：摘要未明确提及局限。