ScholarPulse 日报 2026-09-13

2026-09-13 学术简报：2 篇。LLM judge 在专利撰写任务中可作为有效的迭代优化信号，但其评估与专业律师判断之间存在系统性校准偏差，可靠性高度依赖具体指标。

今日速览

序号	标题	来源	日期	主题	推荐等级
1	Vibe Patenting: Evaluating LLM Judges for Professional Patent-Drafting Agents	arXiv	2026-09-11	AI-Agent	高
2	Toward Self-Adaptive Physical AI: Can LLM Agents Manage Long-Horizon Physical Tasks?	arXiv	2026-09-11	AI-Agent	高

重点论文与技术动态

1. Vibe Patenting: Evaluating LLM Judges for Professional Patent-Drafting Agents

来源：arXiv
日期：2026-09-11
作者/机构：Toshiaki Koike-Akino, Vlad Blaykhman, Ye Wang, Jing Liu, Gene V. Vinokur
主题标签：AI-Agent, arXiv
推荐等级：高
分类：cs.AI

一句话结论

LLM judge 在专利撰写任务中可作为有效的迭代优化信号，但其评估与专业律师判断之间存在系统性校准偏差，可靠性高度依赖具体指标。

核心内容

在 Vibe Patenting 测试平台上，judge 引导的迭代修改持续提升评估质量，而无引导修改则趋于饱和。
迭代 judge 反馈使低推理成本 agent 的性能接近高推理成本 agent，显著降低了达到同等质量所需的计算开销。
更强的模型和更高的推理深度通常提升撰写质量，领域特定的 agentic 工作流可带来额外增益。

方法与数据

构建端到端专利撰写测试平台（Vibe Patenting），由独立调用的 LLM judge 对草稿进行结构化评估并反馈修改。
摘要未明确具体使用的模型列表、发明数量及数据集规模。

价值判断

值得关注：首次将 LLM judge 的可靠性问题置于专利撰写这一高专业度场景中系统验证，对 AI 辅助专业文档生成有直接参考价值。
可复用点：judge 引导迭代修改的范式可迁移至其他需要专业评估的生成任务（如法律文书、技术报告）。
局限/待核查：judge 与律师的一致性”强烈依赖指标”，具体哪些指标一致、哪些不一致摘要未展开；校准偏差的方向和幅度需进一步核实。

摘要

arXiv:2609.13422v1 Announce Type: new Abstract: LLM judges are increasingly used to evaluate and improve AI-generated outputs, yet their reliability for complex professional work remains unclear. We study this problem through Vibe Patenting, an end-to-end patent-drafting testbed for AI-agent evaluation. A separately-invoked LLM judge evaluates generated patent drafts and provides structured feedback for iterative revision. Across multiple inventions and drafting-agent configurations, judge-guided revision consistently improves judge-assessed quality, while unguided revision tends to saturate. Notably, iterative judge feedback enables a low-reasoning agent to approach the performance of a substantially more expensive high-reasoning agent. Stronger models and increased reasoning generally improve judge-assessed drafting quality, while domain-specific agentic workflows provide further gains. We validate the judge against independent evaluation by a professional patent attorney and find meaningful but strongly metric-dependent agreement and systematic calibration differences. These results highlight both the utility and limitations of LLM judges as evaluators and optimization signals for complex professional workflows.

2. Toward Self-Adaptive Physical AI: Can LLM Agents Manage Long-Horizon Physical Tasks?

来源：arXiv
日期：2026-09-11
作者/机构：Varun Kaushik, Yayun Tan, Xiaofan Yu
主题标签：AI-Agent, arXiv
推荐等级：高
分类：cs.AI

一句话结论

零样本LLM多智能体框架在农业长周期任务中可达到与RL智能体相当的管理效果，且在环境发生偏移时展现出更强的适应能力。

核心内容

探索零样本LLM智能体在无人干预下自主管理长周期物理任务的可行性
设计整合规划、工具调用、环境观察与结果验证的多智能体协作框架
在农业任务中与RL智能体进行对比实验，验证不同天气模式下的管理效果与自适应能力

方法与数据

多智能体框架（规划+工具调用+观察+验证），零样本运行，无需大量数据或重训练
农业任务场景，以不同天气模式模拟环境变化，与RL智能体进行对比评估

价值判断

值得关注：为零样本物理AI提供了初步可行性证据，开辟了无需重训练即可适应环境变化的新路径
可复用点：规划-观察-验证的模块化多智能体架构，可迁移至其他物理任务场景
局限/待核查：仅验证了农业单一场景，真实物理环境的复杂性、安全性及长期稳定性未充分讨论

摘要

arXiv:2609.13436v1 Announce Type: new Abstract: Large Language Model (LLM) agents offer a promising path toward autonomously managing long-term physical tasks without human intervention. However, physical tasks require agents to continuously observe the environment, make consequential actions, and remain effective as the environment changes. Existing approaches either require substantial data and retraining, or primarily focus on agents operating in the virtual world. In this work, we explore the feasibility of building a self-adaptive physical AI agent that manages long-term physical tasks in a zero-shot manner and adapts to environmental changes without human intervention. We design a multi-agent framework that integrates planning, tool calling, observation, and verification, and evaluate it on agricultural tasks against reinforcement learning (RL) agents under different weather patterns. Our results show that zero-shot LLM agents can achieve comparable management outcomes to RL agents under the same weather pattern and adapt more effectively than RL when evaluated under a shifted environment, highlighting a promising path toward self-adaptive physical AI agents.