ScholarPulse 日报 2026-07-31

2026-07-31 学术简报：2 篇。AskChem通过声明中心化基础设施显著提升化学文献合成的效率与准确性。

今日速览

序号	标题	来源	日期	主题	推荐等级
1	AskChem: Claim-Centered Infrastructure for Chemistry Literature Synthesis	arXiv	2026-07-30	MCP	高
2	ORCA-bench: How Ready Are Language Model Agents for Oncall?	arXiv	2026-07-30	AI-Agent	高

重点论文与技术动态

1. AskChem: Claim-Centered Infrastructure for Chemistry Literature Synthesis

来源：arXiv
日期：2026-07-30
作者/机构：Bing Yan, Gregory Wolfe, Stefano Martiniani, Kyunghyun Cho
主题标签：MCP, arXiv
推荐等级：高
分类：cs.CL, cs.AI, cs.IR, cs.LG

一句话结论

AskChem通过声明中心化基础设施显著提升化学文献合成的效率与准确性。

一段话：该系统将文献检索单位从论文转向原子化、类型化的声明，每个声明由源DOI和原文引用或证据定位器锚定，支持多维度检索与合成，大幅减少人工操作。

核心内容

将检索单位从论文改为带有来源的原子声明，每个声明由源DOI和原文引用或证据定位器支撑。
提供稳定化分面分类法、证据图和探索性活分类法，实现结构化检索与跨论文合成。
索引240万声明来自14.7万篇论文，支持Web界面及REST/SDK/MCP接口供AI代理使用。

方法与数据

方法：将每篇论文转换为原子、类型化声明，每个由源DOI和证据定位器锚定。
数据：索引2.4M claims from 147K papers。

价值判断

值得关注：解决化学文献合成中手动组装信息的痛点，提升AI代理效率（如GPT-5.5阅读器DOI可解析率达100%）。
可复用点：声明中心化架构可推广至其他科学领域文献处理。
局限/待核查：摘要未明确。

摘要

Chemistry literature synthesis often requires assembling specific findings scattered across many publications, yet existing literature-search systems primarily return ranked document lists. As a result, scientists and AI agents need to locate relevant information, verify their provenance, and assemble cross-paper answers manually. We present AskChem, a claim-centered infrastructure for cross-paper chemistry search. AskChem changes the unit of retrieval from the paper to the provenance-carrying claim: each paper is converted into atomic, typed claims, each grounded by a source DOI and a verbatim quote or an explicit evidence locator. Over this shared claim store, AskChem exposes complementary structures for search and synthesis: a stabilized faceted taxonomy for hierarchical retrieval and browsing, an evidence graph linking claims through relations, and an exploratory living taxonomy that situates indexed papers under scientific principles. AskChem currently indexes 2.4M claims from 147K papers and provides a web interface, as well as REST, SDK, and MCP access for AI agents. On AskChem-Bench, grounding a GPT-5.5 reader in AskChem yields 100% resolvable DOIs, compared with 88.3% without retrieval, and the highest citation density among five tested systems. AskChem is live at https://askchem.org.

2. ORCA-bench: How Ready Are Language Model Agents for Oncall?

来源：arXiv
日期：2026-07-30
作者/机构：Albert Gong, Kyuseong Choi, Abhineet Agarwal, Jason Schechner, Ryan Huang, Raj Agrawal
主题标签：AI-Agent, arXiv
推荐等级：高
分类：cs.CL, cs.AI, cs.SE

一句话结论

语言模型代理在真实oncall根因分析任务中表现有限，最佳准确率仅25.3%，表明需大幅工程改进才能安全用于生产环境。

核心内容

ORCA-bench 基准测试模拟生产级oncall环境，使用实时OpenTelemetry仪器化微服务系统暴露六天指标、日志和跟踪数据（通过Prometheus、Jaeger、OpenSearch via Grafana），并由专家SREs验证症状。
五种前沿代理在中等难度任务上平均准确率25.3%，困难任务仅10.0%，最弱模型在40%报告中产生幻觉根因。
源代码访问对性能至关重要，移除后所有指标显著下降，凸显代码理解在RCA中的核心作用。

方法与数据

基于六天实时生产数据（50 GB测试集）和1,079个系统化RCA任务（覆盖报告具体性、检测时间、共发故障场景）。
摘要未明确具体代理模型名称及评分细节。

价值判断

值得关注：代理在真实oncall场景中低准确率揭示了当前技术与生产可靠性需求的显著差距。
可复用点：ORCA-bench基准测试可被复用于评估其他语言模型代理在系统可靠性任务中的性能。
局限/待核查：测试环境规模有限（50 GB/六天），真实系统更大、更动态、更独特，因此报告的差距是工程投资的保守下限。

摘要

Large language models can write, patch, and search code, but oncall root cause analysis (RCA) demands something different: reasoning over noisy metrics, logs, traces, and source code, starting from ambiguous user-facing reports, often hours after the incident began. We introduce ORCA-bench, a benchmark that puts general-purpose coding agents in a production-fidelity oncall setting. ORCA-bench pairs a live OpenTelemetry-instrumented microservice system--exposing six days of metrics, logs, and traces through real telemetry interfaces (Prometheus, Jaeger, and OpenSearch via Grafana) and full source-code access--with 1,079 RCA tasks that systematically vary report specificity, time-to-detection, and co-occurring fault scenarios. Ground-truth symptoms are curated and signed off by expert SREs, and our LLM-as-judge is independently re-scored by humans (Cohen's $κ_w=0.90$). Across five frontier agents, the best RCA Accuracy is 25.3% on Medium-difficulty tasks (the realistic-input setting) and 10.0% on Hard--a gap that remains even with Claude Fable 5. The weakest model hallucinates an implausible root cause in 40% of incident reports, and removing source-code access degrades every metric. Crucially, these are performances on a curated 50 GB / six-day testbed with tasks investigated in isolation on a system whose code and instrumentation are public. Since real production systems are order of magnitudes larger, more dynamic, and more idiosyncratic, the gap we report is a lower bound on the engineering investment required before frontier coding agents can be safely entrusted with production reliability. We release the public set at https://hub.harborframework.com/datasets/orca-bench/ORCA-bench.