Skip to content
AI NEWS JOURNAL
AI, IN PERSPECTIVE

Search AI news

Search headlines, summaries and articles to connect the dots.

Newest first

Introducing eve 1.0

Vercel announces the general availability of eve 1.0, an open-source framework for building production-ready agents. The release includes performance improvements, expanded integrations, enhanced observability via OpenTelemetry, and new features like multi-tenant support, persistent memory, multiplayer sessions, and lower workflow costs. eve enables developers to build agents with customizable workflows, native UI integrations with frameworks like Next.js, and secure, scalable deployments on Vercel. Early adopters report significant gains in development speed, reduced codebases, and lower AI costs.

Agents are users, so your software needs evals

Agents interact with software through a funnel similar to humans: Discovery, Selection, Access, and Success. To measure whether agents complete user tasks correctly, end-to-end (E2E) evaluations are essential. These tests run multiple trials, score outcomes, and help identify failures across steps like searching cafes, checking budgets, and placing orders. Using frameworks like Harbor, teams can define tasks with specific instructions, environments, verifiers, and known-good solutions. Metrics such as success rate, Pass@k, and Pass^k reveal how reliably agents perform, while trace analysis uncovers root causes of failure. The process includes setting a goal based on observability, customer feedback, or roadmap needs; defining measurable metrics; designing task sets covering common and edge cases; running evaluations at scale using tools like Vercel Sandbox and AI Gateway; and analyzing traces to iterate improvements. Automated trace analysis enables scaling, and repeated evaluation supports continuous improvement, regression detection, and product optimization.

WTF are evals? How to test agents and the software they use

Agents interact with software through a funnel similar to humans: Discovery, Selection, Access, and Success. To measure whether agents complete user tasks correctly, end-to-end (E2E) evaluations are essential. These evaluations assess the full sequence of steps an agent takes, ensuring outcomes meet user requirements. Using frameworks like Harbor, developers can design tasks with defined environments, verifiers, and known-good solutions, then run trials across variants to measure success rate, pass@k, pass^k, efficiency, time, and cost. Analyzing traces reveals root causes of failures, enabling targeted improvements. Iterative evaluation helps catch regressions, test changes, and guide product development, turning evaluations into ongoing infrastructure for building better agent-driven products.

Meet the Sad Wives of AI

Article URL: https://www.wired.com/story/meet-the-sad-wives-of-ai/ Comments URL: https://news.ycombinator.com/item?id=50438429 Points: 27 # Comments: 13

Wake me up when AI can...

Article URL: https://stoppels.ch/2026/10/03/wake-me-up-when-ai-can.html Comments URL: https://news.ycombinator.com/item?id=50440657 Points: 26 # Comments: 7

Ampersand from Humans& now available on AI Gateway

Ampersand, a model designed for group conversations, is now available for free on AI Gateway until November 13, 2026. Built on Z.ai's open-weight GLM 5.3 and trained via reinforcement learning in simulated long-term group interactions, Ampersand understands shared goals, manages disagreements, and decides when to contribute or remain silent. It supports actions like sending messages or passing, and can be integrated via the AI SDK, OpenAI-compatible API, or coding agents using the model name 'humansand/ampersand-v0.1'.