今日速览
| 序号 | 标题 | 来源 | 日期 | 主题 | 推荐等级 |
|---|---|---|---|---|---|
| 1 | Agentic coding without the cloud: evaluating open-weight large language models on longitudinal data preparation tasks | arXiv | 2026-07-23 | RAG | 高 |
| 2 | PATS: Policy-Aware Training Scaffolding for Agentic Reinforcement Learning | arXiv | 2026-07-23 | AI-Agent | 高 |
重点论文与技术动态
1. Agentic coding without the cloud: evaluating open-weight large language models on longitudinal data preparation tasks
- 来源:arXiv
- 日期:2026-07-23
- 作者/机构:Mack Nixon, Liam Wright, Yevgeniya Kovalchuk, Alison Fang-Wei Wu, Martin Danka, Andy Boyd
- 主题标签:
RAG,arXiv - 推荐等级:高
- 分类:cs.AI, cs.CL
一句话结论
本地部署的开源大语言模型在纵向研究数据准备任务中表现优异,平均任务完成率达87.9%,为治理受限研究提供可行路径。
核心内容
- 研究聚焦纵向人口研究数据准备瓶颈,构建开源框架评估AI代理效能,涵盖类别调和与多波合并等任务定义。
- 框架基于英国队列研究六波数据清洗脚本,包含自动化评估工具验证LLM生成的R代码及输出数据。
- 31-35B参数模型在20个任务(创建102变量)中平均完成率达87.9%,接近基准上限,证明消费级硬件可行性。
方法与数据
- 数据集源自英国队列研究清洗脚本,任务定义明确涵盖数据准备关键操作。
- 摘要未明确其他数据细节。
价值判断
- 值得关注:避免敏感数据外传,符合研究治理要求,解决云端传输约束。
- 可复用点:框架开源(GitHub链接),可直接复用于其他治理受限研究的数据准备任务。
- 局限/待核查:摘要未明确提及具体局限。
摘要
Large language models (LLMs) and agents are now widely used tools in code development, with data typically sent to third-party cloud-based models. Their adoption in research using personal data is constrained by governance requirements that typically prohibit data transmission to external services. Locally deployable open-weight models offer an alternative since sensitive data never leave the local environment. We introduce an open-source framework for evaluating the efficacy of AI agents powered by open-weight LLMs on one of the most persistent bottlenecks in research on longitudinal population studies: data preparation. The framework comprises: a curated ground-truth dataset (cleaning scripts preparing six sweeps of data from a British cohort study), task definitions encompassing tasks such as category harmonization and multi-wave merging, and automated routines for evaluating the LLM-produced R code and outputted data. We benchmark LLMs across the (consumer grade) deployment spectrum to assess their efficacy in 20 data preparation tasks (creation of 102 variables). Current state-of-the-art, 31-35B parameter models almost saturated our benchmark ("average task completion" up to 87.9%). The performance of open-weight LLMs running on consumer-grade hardware shows promise of a viable path toward AI-assisted data preparation in governance-restricted research settings. Our framework is publicly available at: https://github.com/UCL-ARC/RRBench.2. PATS: Policy-Aware Training Scaffolding for Agentic Reinforcement Learning
- 来源:arXiv
- 日期:2026-07-23
- 作者/机构:Yipeng Shi, Zhipeng Ma, Yue Wang, Qitai Tan, Yang Li, Peng Chen
- 主题标签:
AI-Agent,arXiv - 推荐等级:高
- 分类:cs.AI
一句话结论
PATS通过政策感知动态训练脚手架显著提升了长时域LLM代理强化学习的性能。
核心内容
- 弱策略在长时域RL中重复失败,产生无信息轨迹,限制有效优化。
- PATS将技能重构为动态训练脚手架,将策略轨迹转换为证据卡,通过任务特定评估动态调整后续上下文。
- 随策略改进,移除冗余上下文以减少依赖,优化策略使用环境奖励,部署时丢弃脚手架。
方法与数据
- 方法:动态调整上下文以适应策略进化,使用任务特定评估优化轨迹。
- 数据:摘要未明确具体数据集细节。
价值判断
- 值得关注:在ALFWorld和WebShop上性能提升最高18.6%,在七个搜索增强QA基准上减少32.1%提示令牌。
- 可复用点:动态训练脚手架框架可推广至其他代理强化学习任务。
- 局限/待核查:摘要未明确局限。