Skip to content
AI NEWS JOURNAL
AI, IN PERSPECTIVE

Search AI news

Search headlines, summaries and articles to connect the dots.

Newest first

Supercharging Olmo-core for Efficient and Scalable MoE Training

arXiv:2611.18956v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models add parameters without proportionally increasing per-token compute. However, training cost also depends on bytes moved, memory held, and kernels launched, and many of those costs follow total parameter size rather than active compute. On infrastructure built for dense models, this mismatch is costly: in early baselines, throughput fell as stored expert parameter size grew, even though the useful work per token stayed nearly fixed. To close this gap, we present a redesigned training stack in Olmo-core, built for efficient, scalable MoE training. We replace the Fully Sharded Data Parallel (FSDP) of earlier Olmo-core versions with an architecture built on Distributed Data Parallel (DDP), and add Expert Parallelism (EP), Pipeline Parallelism (PP), a distributed optimizer, and activation recompute. We propose NVSHMEM-based rowwise EP, which writes each token directly into its expert's buffer, avoiding dispatch and combine permutation overhead. We make EP fully synchronization-free by keeping routing metadata on the GPU and using device-scheduled grouped GEMMs. Closely integrated MXFP8 cuts compute, activation-memory, and transport costs, and topology-agnostic checkpoints let the parallel layout change between runs. On NVL8 B300 nodes, the stack supports training configurations from 12.9 billion to 1.2 trillion total parameters on up to 512 GPUs, reaching 858 useful-model TFLOP/s/GPU at 1.2 trillion with MXFP8 and per-layer recompute; an experimental capacity test with the optional DeepEP v2 backend reaches 2.38 trillion. We also explain each design choice and share findings for other training systems; for example, we identify a failure pattern we call Token Gerrymandering, in which the MoE router can learn to hack the load-balancing loss, and find that overlapping communication with computation can slow the overlapped kernels by more than it hides.

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly helping developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

Agents are users, so your software needs evals

Agents interact with software through a funnel similar to humans: Discovery, Selection, Access, and Success. To measure whether agents complete user tasks correctly, end-to-end (E2E) evaluations are essential. These tests run multiple trials, score outcomes, and help identify failures across steps like searching cafes, checking budgets, and placing orders. Using frameworks like Harbor, teams can define tasks with specific instructions, environments, verifiers, and known-good solutions. Metrics such as success rate, Pass@k, and Pass^k reveal how reliably agents perform, while trace analysis uncovers root causes of failure. The process includes setting a goal based on observability, customer feedback, or roadmap needs; defining measurable metrics; designing task sets covering common and edge cases; running evaluations at scale using tools like Vercel Sandbox and AI Gateway; and analyzing traces to iterate improvements. Automated trace analysis enables scaling, and repeated evaluation supports continuous improvement, regression detection, and product optimization.

AI Gateway now supports Private Inference

Vercel's AI Gateway now enables enterprises to integrate private, self-hosted, or prerelease models through a unified interface, offering consistent authentication, SDKs, APIs, and observability. Users can switch between private and catalog models using the same endpoints, keep models scoped to their team with encrypted credentials and optional Zero Data Retention, connect to coding agents like Claude Code and Cursor, monitor traffic alongside public models, configure fallbacks to catalog models, and manage providers via CLI. Private model requests are logged alongside catalog requests for unified observability.

AI made integrations the vendor’s problem

In the second iteration of the AI Agent Development Tools report, I replaced the integrability axis with enterprise-readiness. I wanted this to signify the departure from 2020-style IPaaS. With MCP getting traction to the point where every vendor offers an MCP server, I said the integration burden now

An Operationalization of Opaque Serial Depth

arXiv:2611.17495v1 Announce Type: new Abstract: Chain-of-thought (CoT) is currently a valuable tool for overseeing AI models, but some architectural shifts could significantly reduce CoT monitorability. We operationalize a measure that serves as a proxy for the amount of unverbalized serial cognition a model can perform, as a specific instantiation of the notion of opaque serial depth: roughly, the length of the longest path through a model's computation that does not pass through an interpretable bottleneck. Our main contribution is a particular standard for what counts as such a bottleneck. We define natural-language-rooted (NL-rooted) nodes as nodes that were initialized from a pre-training prior to produce human-understandable outputs, whose output space has not been expanded beyond that prior, and whose outputs the model has not been modified to interpret as a different type of data. By construction, the resulting measure, NLS depth, can be adjudicated before training from an architecture, training recipe, and deployment configuration alone, and is insensitive to ordinary post-training such as prompting, distillation, and RL. We find that, across open-weight models, NLS depth correlates strongly with layer count, with some exceptions, and has grown slowly over recent years. It would likely grow slowly under continued scaling of standard transformers, while latent-reasoning architectures such as full-bandwidth transformers could increase it by a large amount. We work through how NLS depth applies to a range of existing and proposed architectures, discuss alternative operationalizations, and give a worked example bounding the depth of Gemma3-1B. We argue that AI developers should track and report the NLS depth of their models as a complement to direct measurements of CoT monitorability.

b12431

The update addresses issues with reasoning effort levels by ensuring rejected levels are properly handled during parsing, preventing 500 errors on API endpoints. It maintains support for template-ignored levels and aliases while trimming the output_config.effort test to a single level.

Can You Trust Your Finance Agent? Drift, Diagnosis and Auditable Intervention on One Shipped Financial Research Agent

arXiv:2611.17939v1 Announce Type: new Abstract: Can you trust a finance agent you did not build? We take the reference agent shipped with the Vals Finance Agent Benchmark v2, leave it untuned, and ask four questions a leaderboard score does not answer. Is the score stable? Seven identical runs on 18 questions pass between 105 and 123 of 149 rubric criteria, and the bill varies by up to 17%. When the agent fails, who owns the fix? We classify 20 failed trajectories, from three runs of nine held-out questions, by the layer whose owner would remedy them, under a taxonomy frozen in advance: 13 belong to the agent's instructions, 3 to its tools, 4 to the model, none to its data. Can that diagnosis be trusted? Four blind readers from three model families, given only the traces, name the same fault as our LLM analyzer in 94% of cases, the rate at which they agree with one another (93%). Can a change be trusted? An optimization loop reads the traces, makes one targeted change per round and keeps it only if it clears a fixed threshold. Under a cost objective its final configuration costs 0.34x the baseline on the held-out questions (95% CI [0.22, 0.58]) at accuracy within noise. Under an accuracy objective it gains 14 points on the development questions and 1.5 on the held-out ones (CI [-5.3, +7.2]). Judged against the measured run-to-run noise, under a rule written before the test was opened, the first is a real saving and the second is not. This is a case study of one agent and nine test questions, not a general result.

Customize answers per tenant in Amazon Bedrock Managed Knowledge Base

Retrieval instructions in Amazon Bedrock Managed Knowledge Base set the voice, format, retrieval guidance, and answer policy for each tenant on shared knowledge bases. This post shows how to give each tenant its own answers, keep tenant data apart with metadata filters, scope access with IAM tags, and roll out changes one tenant at a time.

QCAP-VQA: Question-Conditioned Capped Prior-Mass Supervision for Medical Visual Question Answering

arXiv:2611.17708v1 Announce Type: new Abstract: Medical visual question answering models can answer correctly without attending to the anatomy that the question names. Automatic region priors allow spatial supervision without expert masks, but they are noisy, and it is unclear how strongly they should shape attention and how their effect should be verified. We present QCAP-VQA, which supervises a subset of cross-attention heads with a question-conditioned prior built from ranked segmentation proposals. Its capped prior-mass objective penalizes insufficient attention inside the prior only below a threshold. We evaluate attention against SLAKE structure boxes that play no role in training, separating attention on the named anatomy from how attention shifts between questions about the same image. QCAP-VQA matches the accuracy of a baseline without spatial supervision (84.26% vs. 84.23%), and every evaluated prior-supervised model places more attention on the annotated structures. Over a 64-fold range of regularization strength, the capped objective keeps prior mass between 0.67 and 0.77, whereas a pooled region-mass objective drives it to 0.97 and loses accuracy. The cost lies in question conditioning: as the constraint widens or strengthens, attention shifts agree less with the expert-defined contrast. Across 13 configurations, prior-referenced and expert-referenced evaluation rank the models in largely opposite order (Spearman correlation of -0.87), and the capped models it favors track the prior's own errors. Because clinical use of medical visual question answering requires answers grounded in the queried anatomy, supervision from automatic priors should be judged against independent anatomical references.

Qwen4-Max now available on AI Gateway

Qwen4-Max, the first model in Alibaba's Qwen4 series, is now available on AI Gateway. It features a 1M-token context window, native vision capabilities, and excels in coding, professional tasks, and long-running agent workflows. The model can generate and test 3D scenes in Blender and games in Unity, Unreal Engine, and Godot, while also handling complex document analysis for finance, legal, and healthcare use cases. It produces cited reports, spreadsheets, and slide decks, and supports iterative improvements via a Live Delivery plugin. Compared to previous models, it uses fewer tokens, writes more secure code, and integrates with tools like Claude Code, Codex, and Cursor through Vercel CLI setup. AI Gateway offers unified API access, cost tracking, Zero Data Retention, budget controls, and no markup pricing, including for BYOK requests.

RaFL: An Empirical Study of Random Filter Losses for Image Reconstruction

arXiv:2611.17707v1 Announce Type: new Abstract: Pixel losses train image reconstruction networks toward a point statistic of the target distribution, often averaging away fine textures under uncertainty. Perceptual losses alleviate this issue using pretrained features, but those features are inherently domain-specific. We study a random filter loss (RaFL) that compares prediction and target through filters with random weights, avoiding both pretrained features and task-specific loss engineering. RaFL combines random convolutions, random Fourier features, and random wavelet filters, automatically balancing their contributions through gradient-norm matching, eliminating manual loss-weight tuning and grid search. We systematically investigate three design choices: filter family, weight resampling schedule, and weight initialization. Based on validation PSNR and stability across random seeds, we select the configuration that uses all three filter families, redraws filters every optimization step, and employs Gaussian initialization. We evaluate RaFL on three 2D and two 3D image reconstruction tasks. RaFL outperforms a pixel loss on three tasks and matches it on two, and demonstrates stable performance across random seeds on all five tasks. RaFL adds less than 2% to the backbone's forward FLOPs and no inference cost, requiring $182\times$ fewer loss FLOPs than a VGG19 perceptual loss. These findings suggest that RaFL is a simple, tuning-free, and domain-agnostic loss objective for 2D and 3D image reconstruction.

Show HN: 1MB Tenstorrent driver with ptx like kernels

The world of AI used to have people training models for solving substantial issues - like beat world's best go players or solving protein folding. Now that the AI got good, perhaps we forgot a bit how to do it. There is still cheap hardware around for tinkerers and people who want to continue good AI legacy. But the hardware may miss software. It is for those people that I started working on zyx long time ago - ML library in hopes of running on any hardware. One example of such hardware is tenstorrent. But the basics must be built first. And so today I'm showing you skamandros. A tiny, self-contained driver and compiler for tenstorrent cards (tested on p100a). PTX like input, mostly abstracted away hardware details (other than tiling), automatic optimizations and operation fusion. Hope you can use this software to write AI models that people can like. Comments URL: https://news.ycombinator.com/item?id=50509698 Points: 1 # Comments: 0

Show HN: atl, a Rust CLI for Jira, Confluence and Bitbucket with ~15ms overhead

Hello HN,I built `atl`, a Rust CLI for Jira, Confluence, Bitbucket, Jira Service Management, Trello and Atlassian Admin. The same commands work on Cloud and Data Center (on-prem).It's built for both people and AI agents: an --agent mode that outputs JSON and never prompts, a built-in MCP server with 136 tools, and a read-only login so an agent can't change anything. Confluence pages can be edited as Markdown and kept in git.It has SDK support as well for Python, TypeScript/JS, and WASM.Any feedback and contributions are welcome! Comments URL: https://news.ycombinator.com/item?id=50509662 Points: 1 # Comments: 0

Show HN: Comemo – composite memory provider for Hermes Agent

There are many memory providers that can be used externally with Hermes Agent. However, like many other AI agents, Hermes only provides one slot for an external memory provider.Comemo takes that slot and acts as a composite provider. How? It fans out memory writes to multiple backends such as Mem0, Honcho, and Supermemory. On retrieval, it queries the enabled providers, merges their results, and removes duplicates before returning them to Hermes.I intend to add support for more memory providers and eventually extend Comemo to other open-source AI agents with similar limitations. Comments URL: https://news.ycombinator.com/item?id=50518538 Points: 1 # Comments: 0