我的收藏提交工具
FOXAI SEARCH

全站搜索

新闻报道 446

Build a daily dependency watcher with LangGraph and Amazon Bedrock AgentCore web search

The Web Search API in Amazon Bedrock AgentCore gives agents current, cited web results signed with the AWS credentials they already use. In this post, build a daily dependency watcher as a LangGraph graph that researches package ecosystems in parallel, keeps its rules in code, and checks every finding against its cited sources.

翻译服务暂未运行,当前显示原文。
AWS Machine Learning Blog

ForeSpark: Joint Draft and Verification Budget Scheduling for Parallel Speculative Decoding

arXiv:2611.13442v1 Announce Type: new Abstract: Parallel drafters accelerate LLM decoding by proposing a whole block of draft tokens in a single forward pass, making them attractive for LLM serving. Recent systems optimize goodput (i.e., accepted tokens per second) because rejected drafts steal compute from other requests. However, these systems control only how many tokens are verified, so every request still drafts a full block. At high serving load, we find that drafting consumes up to 40% of step time, of which a majority is discarded because it is never verified. Yet, draft-length control from autoregressive drafters does not carry over, because a parallel drafter commits to its length before its single pass. Moreover, draft and verification lengths are coupled across the batch, so they must be selected jointly for every request while keeping the decision computationally cheap at every step. In this paper, we propose work-conserving speculation (WCS), a design that jointly allocates draft and verification budgets per request in LLM serving, and present ForeSpark as its prototype. ForeSpark realizes WCS with (i) a context-aware pre-draft estimator that predicts acceptance for every candidate length from the target model's context features, (ii) a lightweight joint draft-verify scheduler that selects both lengths through a global sort, and (iii) a post-draft budget distribution that spends verification on the most promising tokens. Implemented on vLLM, ForeSpark improves throughput by up to 1.77x over the state-of-the-art DSpark and by 2.49x over autoregressive decoding at a matched per-request generation rate, without changing the output distribution. It also tolerates higher load, sustaining up to 1.51x the request rate of DSpark under latency service level objectives (SLOs).

翻译服务暂未运行,当前显示原文。
arXiv cs.DC

Gemini Decision 1 现已在 AI Gateway 上可用

谷歌的 Gemini Decision 1 是一款支持文本、图像、音频、视频和 PDF 的多模态决策模型,现可通过 AI Gateway 访问。它为请求路由、策略检查、工具过滤和结果排序等任务提供布尔概率、选项和评分,而无需生成输出 token。该模型通过 AI SDK、兼容 TypeSafe 的 API 和 Decisions API 提供支持,AI SDK 7 中已支持 PDF、音频和视频输入。决策调用会被记录,并与其他模型请求一起计入预算。

尚无对应语言译文,当前显示原文。
Vercel