Skip to content
AI NEWS JOURNAL
AI, IN PERSPECTIVE

Search AI news

Search headlines, summaries and articles to connect the dots.

Newest first

Blog Review: Oct. 28

STCO for heterogeneous design; physical AI interactions; signing off AI-generated designs; workload profile analysis. The post Blog Review: Oct. 28 appeared first on Semiconductor Engineering.

新闻源配图 · SemiEngineering
Timeline and related coverage

DualPace: Decoupled Scheduling and Cooperative GPU--CPU Decoding for LLM Serving

arXiv:2610.33501v1 Announce Type: new Abstract: Large language model (LLM) serving relies on batching to improve inference throughput, but the key-value (KV) cache grows with sequence length and limits GPU-resident concurrency. CPU offloading can expand KV capacity using host memory, yet existing systems remain largely GPU-gated: CPU-resident requests make progress only when the GPU scheduler exposes suitable overlap opportunities. As a result, host memory and CPU compute cannot act as an independent serving resource, leading to bursty CPU-side progress and inflated tail inter-token latency (ITL) under load. We present DualPace, an LLM serving runtime that turns the host into an independently scheduled decode engine. DualPace separates scheduling authority across two engines: the GPU engin

Timeline and related coverage