今日速览
| 序号 | 标题 | 来源 | 论文日期 | 主题 | 推荐等级 |
|---|---|---|---|---|---|
| 1 | When the Tool Decides: LLM Agents Defer Blindly to Graph Neural Network Tools, and Stronger Backbones Defer More | arXiv | 2026-06-12 | AI-Agent | 高 |
| 2 | Verifiable User Simulation for Search and Recommendation Systems | arXiv | 2026-06-12 | AI-Agent | 高 |
重点论文与技术动态
1. When the Tool Decides: LLM Agents Defer Blindly to Graph Neural Network Tools, and Stronger Backbones Defer More
- 来源：arXiv
- 论文日期：2026-06-12
- 作者/机构：Zhongyuan Wang, Pratyusha Vemuri
- 主题标签：
AI-Agent,arXiv - 推荐等级：高
- 分类：cs.AI, cs.LG
一句话结论
LLM代理在节点分类任务中盲目遵循GNN工具输出，不进行自主判断，且模型能力越强，依赖程度越高。
核心内容
- 代理在ogbn-arxiv和WikiCS文本属性图上5种子实验中与GNN预测一致率达97.6-99.2%，成为GNN复读机，绕过自身推理。
- 模型能力提升（Qwen2.5 1.5B-7B），代理与GNN同意率从0.60升至0.98，依赖非弱模型问题。
- 即使存在更优替代工具（如邻居标签工具在高同质性场景性能0.81 vs GNN 0.71），代理仍盲目依赖，导致性能损失0.12-0.22。
方法与数据
- 方法：将冻结GNN作为ReAct风格LLM代理的显式工具进行节点分类测试。
- 数据：ogbn-arxiv和WikiCS文本属性图。
价值判断
- 值得关注：评估代理+工具系统时不能假设代理添加判断，需主动设计选择性调用机制。
- 可复用点：简单选择性调用门可恢复高同质性场景性能（从0.71到0.83）。
- 局限/待核查：可靠选择性调用受限于可用信息，最佳门限仅能恢复oracle头差的三分之一。
摘要
A growing line of work equips large language model (LLM) agents with graph neural networks (GNNs) as callable tools, assuming the agent exercises judgment over when and how much to rely on such a tool. We test this directly. We expose a frozen GNN to a ReAct-style LLM agent as an explicit tool and measure, on node classification over a text-attributed graph (ogbn-arxiv, replicated on WikiCS), whether the agent uses the tool or merely obeys it. We find the agent does not exercise judgment: its predictions agree with the raw GNN's 97.6-99.2% of the time (5 seeds), collapsing into a GNN parrot that adopts the tool's output wholesale and bypasses its own reasoning. Sweeping backbone capability (Qwen2.5 0.5B-7B), the deference is not a weak-model artifact: among models able to invoke the tool, agreement rises with capability (0.60 to 0.98 from 1.5B to 7B). Crucially, the cost of deference does not shrink as capability grows and grows where alternatives emerge: a per-node oracle over the available actions beats the parrot by 0.09-0.18 at 3B and 0.12-0.22 at 7B, roughly doubling at high homophily, because the parrot is pinned to the frozen GNN while the agent's alternatives improve; at 7B a simple neighbour-label tool overtakes the GNN at high homophily (0.81 vs 0.71) yet the agent still defers. A simple selective-invocation gate recovers about half of that high-homophily gap (0.71 to 0.83) but yields no net global gain, and held-out estimates bound the best achievable gate over standard test-time features to at most a third of the oracle headroom: reliable selective invocation looks limited by available information, not merely router design. Our results are a cautionary measurement: evaluations of agent+tool systems cannot assume the agent adds judgment on top of the tool, and selective invocation must be designed in rather than expected to emerge from scale.2. Verifiable User Simulation for Search and Recommendation Systems
- 来源：arXiv
- 论文日期：2026-06-12
- 作者/机构：Chenglong Ma, Xinye Wanyan, Danula Hettiachchi, Ziqi Xu, Yongli Ren, Jeffrey Chan
- 主题标签：
AI-Agent,arXiv - 推荐等级：高
- 分类：cs.IR, cs.HC
一句话结论
本教程提出一种设计-审计框架，将用户模拟器视为可验证的工程制品，以解决LLM驱动模拟器的不透明性和潜在偏见问题。
核心内容
- LLM-based用户模拟器常不透明，难以解释选择，且可能因用户背景（如语言、教育水平、文化）产生歧视性响应，影响少数群体公平性。
- 框架包含七个可审计组件：结构化人格、任务感知合同、人机执行匹配、可审计轨迹、人格对齐验证、结构化反馈及更新循环。
- 通过推荐列表评估和搜索查询制定的动手实验，参与者可端到端检查模拟器行为，区分诊断差异分析与统计验证，并应用忠实度、可信度及人口统计偏见检查。
方法与数据
- 框架基于七个可审计组件构建。
- 摘要未明确具体数据来源。
价值判断
- 值得关注：解决模拟器不透明性和偏见问题，促进AI系统对少数群体的公平处理。
- 可复用点：框架组件设计可复用于其他推荐和搜索系统评估。
- 局限/待核查：摘要未提及具体局限。