Skip to content
AI NEWS JOURNAL
AI, IN PERSPECTIVE

Search AI news

Search headlines, summaries and articles to connect the dots.

Newest first

180B Parameters, 32GB of RAM: Pushing MoE Inference to the Limit with Transformers

The author explores streaming MoE (Mixture of Experts) model experts directly from SSD to achieve near-Strata performance on a 32GB MacBook M5, using a custom implementation in Hugging Face Transformers. By optimizing expert caching, leveraging Metal kernels, and minimizing host-GPU synchronization delays, the approach reaches ~14 tok/s—within 6% of Strata's performance—demonstrating that efficient expert streaming is feasible on consumer hardware without C++ extensions.

b12480

The Llama UI design system, including tokens, surfaces, fonts, and base components, has been moved from a temporary repository to ui/lib/ui, where it is now published as @ggml-org/llama-ui. Llama Chat now imports all UI elements—including model components, constants, enums, and helpers—from this centralized library, eliminating duplicate code. The update includes updated branding with Llama Sans and Llama Mono fonts, improved submenu rendering, enhanced elevation for chat form elements, and integration with the llama-brand package for colors and logo. The UI is now consistently applied across the application, with builds and deployment configurations adjusted accordingly.

b12451

The web UI for Llama.cpp has been relocated from tools/ui to ui/apps/chat as part of a structural refactoring to separate the component library (ui/lib) from the applications built on it. The app is now officially named Llama Chat, with its package renamed to @ggml-org/llama-chat, while maintaining private status. Documentation, help text, comments, and other references have been updated to reflect the new name, though legacy identifiers remain unchanged for backward compatibility.

b12475

The Llama Chat UI has been upgraded to SvelteKit 3, integrating its built-in service worker for improved update detection and offline functionality. The migration includes updates to Vite 8, TypeScript 6, Storybook 10, and ESLint 10, along with the replacement of deprecated packages and resolution of npm audit issues through dependency overrides, including pinning minimatch to version 11.0.5 to address a security vulnerability. Changes also align with current Svelte patterns, such as using $state instead of stores and adopting new syntax for actions and class bindings.

How Duco transforms financial data automation with Amazon Bedrock AgentCore

Duco, a data automation platform for tier-one banks, rebuilt its agentic AI stack on Amazon Bedrock and Amazon Bedrock AgentCore. With strict per-tenant isolation, a multi-model strategy, and an MCP skills architecture that cut context overhead by 90%, Duco cut reconciliation deployment time 10x and onboarding costs 70%.

Agents are users, so your software needs evals

Agents interact with software through a funnel similar to humans: Discovery, Selection, Access, and Success. To measure whether agents complete user tasks correctly, end-to-end (E2E) evaluations are essential. These tests run multiple trials, score outcomes, and help identify failures across steps like searching cafes, checking budgets, and placing orders. Using frameworks like Harbor, teams can define tasks with specific instructions, environments, verifiers, and known-good solutions. Metrics such as success rate, Pass@k, and Pass^k reveal how reliably agents perform, while trace analysis uncovers root causes of failure. The process includes setting a goal based on observability, customer feedback, or roadmap needs; defining measurable metrics; designing task sets covering common and edge cases; running evaluations at scale using tools like Vercel Sandbox and AI Gateway; and analyzing traces to iterate improvements. Automated trace analysis enables scaling, and repeated evaluation supports continuous improvement, regression detection, and product optimization.

Four Amazon Bedrock inference choices that affect cost

Your Amazon Bedrock inference bill can keep climbing even after you tune prompts and application logic. Four request-level choices (cache duration, service tier, reasoning effort, and maxTokens) change how many tokens the service processes and the rate applied. We show how we tested each and how to verify the effect.

WTF are evals? How to test agents and the software they use

Agents interact with software through a funnel similar to humans: Discovery, Selection, Access, and Success. To measure whether agents complete user tasks correctly, end-to-end (E2E) evaluations are essential. These evaluations assess the full sequence of steps an agent takes, ensuring outcomes meet user requirements. Using frameworks like Harbor, developers can design tasks with defined environments, verifiers, and known-good solutions, then run trials across variants to measure success rate, pass@k, pass^k, efficiency, time, and cost. Analyzing traces reveals root causes of failures, enabling targeted improvements. Iterative evaluation helps catch regressions, test changes, and guide product development, turning evaluations into ongoing infrastructure for building better agent-driven products.

Show HN: Black Ops II statically recompiled to native ARM64 macOS

I grew up with Black Ops 2 and wanted to see if I could make it work on my 2020 MacBook Pro. So I used Claude Opus 5.5 to create a tool that converts your copy of the game (from Steam) to a native ARM64 Mac app, with almost 60fps in all areas. The whole game is statically recompilated ahead-of-time (AOT) and does not use any emulator whatsoever, be it Wine, Crossover or Rosetta. The best thing is the performance, although some heat issues are still to fix. You get to play the whole Campaign, local Multiplayer (with bots) and local Zombies (all DLCs maps; you need to have them in your Steam library obviously) at about 60fps. Split-screen was not in the PC version, but I've been working on it lately; it's coming soon! LAN mode is on the roadmap for later too :) Comments URL: https://news.ycombinator.com/item?id=50449891 Points: 1 # Comments: 0

Show HN: CodeLens AI-Augmented Code Review Tool

As AI writes more code, the bottleneck has become human-driven code reviews. Code Lens serves to aid that task by augmenting the review outside of line-level AI feedback. Check it out and let me know what you think. Thanks.Try it on a public PR: npx @blong8334/codelens pr https://github.com/expressjs/express/pull/7574 Comments URL: https://news.ycombinator.com/item?id=50446651 Points: 1 # Comments: 0

n8n@2.48.0

The n8n 2.48.0 release addresses multiple bugs, including fixes for AI builder, compression node, core functionality, editor improvements, and RabbitMQ node issues. New features include enhanced assistant models, Microsoft Work IQ OAuth2 credential support, app bindings UI, migration report enhancements, and improved user experience elements like chat message unification and credential modal restrictions.

n8n@3.2.0

The n8n 3.2.0 release introduces several bug fixes addressing issues in AI Builder, core functionality, editor UI, and node-specific behaviors, including improved handling of JSON, XML, CSV, and YAML responses, enhanced security through expanded IP blocking, and better error reporting. New features include support for Microsoft Work IQ OAuth2 credentials, a new App bindings UI, and updated default Assistant models. The release also reverts some previous changes due to compatibility or stability concerns.

Wake me up when AI can...

Article URL: https://stoppels.ch/2026/10/03/wake-me-up-when-ai-can.html Comments URL: https://news.ycombinator.com/item?id=50440657 Points: 26 # Comments: 7