ScholarPulse 日报 2026-06-18

2026-06-18 学术简报：2 篇。AI代理在小分子临床前药理学决策中表现有限，最强模型准确率仅59.3%。

今日速览

序号	标题	来源	日期	主题	推荐等级
1	TxBench-PP: Analyzing AI Agent Performance on Small-Molecule Preclinical Pharmacology	arXiv	2026-06-17	AI-Agent	中
2	Runtime Compliance Verification for AI Agents	arXiv	2026-06-17	AI-Agent	中

重点论文与技术动态

1. TxBench-PP: Analyzing AI Agent Performance on Small-Molecule Preclinical Pharmacology

来源：arXiv
日期：2026-06-17
作者/机构：Hannah Le, Ramesh Ramasamy, Alex Urrutia, Mahsa Yazdani, Tim Proctor, Kenny Workman
主题标签：AI-Agent, arXiv
推荐等级：中
分类：cs.AI, cs.LG

一句话结论

AI代理在小分子临床前药理学决策中表现有限，最强模型准确率仅59.3%。

一段话：TxBench-PP基准测试表明，当前AI代理无法可靠地从真实实验数据中恢复临床前药理学决策，最高准确率仅为59.3%，远低于实用要求。

核心内容

TxBench-PP是首个针对小分子临床前药理学的可验证基准，测试AI代理从真实实验数据而非文献事实中推导结论的能力。
基准包含100个评估，覆盖机制作用、药效学推理、化合物靶点结合等多方面药理学任务。
在16种模型配置测试中，无系统可靠工作，最强模型Claude Opus 4.8 / Pi仅59.3%准确率（178/300）。

方法与数据

基准包含100个评估，基于程序阶段、实验类型和任务结构；代理接收工作流快照，在编码环境中检查文件并返回结构化答案。
测试使用4800个轨迹、11个模型，答案由系统确定性评分。

价值判断

值得关注：AI代理在药物发现中的实际部署需基于真实数据的严格评估，TxBench-PP填补了临床前药理学评估的空白。
可复用点：基准设计和测试流程（工作流快照、结构化答案）可推广至其他药物发现阶段。
局限/待核查：当前AI代理在复杂药理学决策中表现不足，最高准确率仅59.3%，表明技术需重大改进。

摘要

Artificial intelligence (AI) agents promise to accelerate drug discovery by compressing interpretation and decision-making loops, but practical deployment requires trusted evaluation on realistic program decisions. We introduce TherapeuticsBench Preclinical Pharmacology (TxBench-PP), a verifiable benchmark for small-molecule preclinical pharmacology and the first focused slice of a broader TherapeuticsBench effort across drug-discovery stages and therapeutic modalities. TxBench-PP tests whether agents can recover accurate conclusions from real-world assay data rather than memorized facts from literature. The benchmark contains 100 evaluations indexed by program stage, assay type, and task structure, spanning mechanism-of-action (MoA) and pharmacodynamic (PD) reasoning, compound-target engagement, causal target validation, developability and safety, and translational efficacy. Agents receive realistic workflow snapshots, inspect files in a coding environment, and return structured answers graded deterministically. Across 16 model-harness configurations, comprising 11 models and 4,800 trajectories, no system reliably recovered preclinical pharmacology decisions. The strongest configuration, Claude Opus 4.8 / Pi, passed 59.3\% of endpoint attempts (178/300; 95\% CI, 51.1-67.6), followed by GPT-5.5 / Pi at 55.3\% (166/300; 47.0-63.6).

2. Runtime Compliance Verification for AI Agents

来源：arXiv
日期：2026-06-17
作者/机构：Nafiseh Kahani, Masoud Barati, Diana Addae
主题标签：AI-Agent, arXiv
推荐等级：中
分类：cs.SE

一句话结论

C-Trace框架通过实时监控AI代理执行轨迹，有效保障GDPR合规性，在10%提取噪声下将攻击成功率控制在12%以下、假阳性率不超过16%，完美提取时实现0%攻击成功率。

核心内容

将GDPR核心要求（如同意、目的限制、数据最小化及删除权）形式化为执行轨迹上的策略谓词。
部署运行时监控器拦截所有工具调用和模型输出，自动拒绝不合规行为。
利用DSPy生成提示和红队语料库中的攻击对话测试代理，诱导合规违规行为。

方法与数据

基于GDPR重构的四个案例研究进行评估，采用攻击对话测试方法。
摘要未明确具体数据集规模及噪声类型细节。

价值判断

值得关注：解决AI代理运行时合规性验证空白，弥补现有离线测试的不足。
可复用点：实时监控机制和攻击测试方法可迁移至其他数据保护法规框架。
局限/待核查：在10%提取噪声下性能有限，未评估更高噪声或更复杂GDPR条款场景。

摘要

AI agents now handle personal data through tool use, function calls, and multi turn dialogue, which can create obligations under the General Data Protection Regulation (GDPR). Current testing practices mainly rely on offline red teaming or static prompt review, but they do not guarantee at runtime that agent behavior follows regulatory rules. We propose C-Trace (Compliance Trace based Runtime Agent Conformance Enforcement), a verification framework that: (i) expresses a subset of GDPR requirements, including consent, purpose limitation, data minimization, and the right to erasure, as formal policy predicates over agent execution traces; (ii) uses a runtime monitor that intercepts every tool invocation and model output and rejects non-compliant actions; and (iii) tests the agent with attack dialogues, including DSPy generated prompts and verbatim prompts from red teaming corpora, that try to induce violations. We evaluate the framework on four case studies reframed to GDPR. Under 10 percent per-category extractor noise, including drop-out and over-typing, the monitor keeps the attack success rate at less than or equal to 12 percent, below the baselines we compare against, and false positives at less than or equal to 16 percent, and reaches 0 percent ASR under perfect extraction.