Search AI news

FusionEval Released: A New Benchmark for Large Language Model Fusion

On October 24, 2026, researchers published the paper arXiv:2610.30162v1 on arXiv, introducing FusionEval—the first unified benchmark specifically designed to evaluate fusion methods for large language models. It covers vulnerable aspects such as reasoning chain length and tool invocation, and supports comparisons between fine-tuning-based and zero-shot fusion approaches.

Why it matters：This benchmark fills the gap left by existing evaluations, which inadequately cover long-chain reasoning and tool usage; it clearly distinguishes between average performance and worst-case capability retention, provides a reproducible expert pool (based on Qwen3/Llama), multi-dimensional metrics (including GPU memory and runtime), and open-source code, offering strong practical guidance.

Research reveals the “norm–oracle gap” problem in benchmarking coding agents.

A paper published on arXiv on October 24, 2026, points out that current mainstream benchmarks for coding AIs—such as SWE-Milestone—suffer from a “specification–oracle gap”: the natural-language descriptions of tasks fail to capture crucial implementation details required for testing (e.g., constant names, module paths, and precise error messages), resulting in evaluations that measure the ability to reproduce upstream code rather than genuine understanding and reasoning.

Why it matters：This study is the first to formally define this type of bias, develop a static analysis tool covering five languages (with 89.2% precision and 92.9% recall), and empirically demonstrate in SWE-Milestone that 25 milestone tasks contain non-inferable implicit dependencies, significantly increasing the failure rate (80.6% vs. 48.4%). These findings reveal that current leaderboards may systematically favor “name‑copying” agents. The results have been made publicly available, including the open-source auditing tool and corrective resources, offering both methodological and practical value.

Recursive releases Flint: a lightweight, AI-driven programming model that can run locally.

AI startup Recursive has unveiled Flint, an open-source coding model that it claims was developed primarily using its in-house AI research system, Eureka Machine. With just 40 billion parameters and only 2.5 billion activations in practice, the model can run offline on a standard laptop—such as an M4 Pro MacBook—at an inference speed of 131 tokens per second, outperforming most models of comparable size on benchmarks like SWE-Bench.

OpenAI's latest GPT-6 series models will be available on Google Cloud and the Antigravity platform.

OpenAI’s cutting-edge GPT-6 and 6.1 series models (Astra, Sol, Luna) will be launched on Google Cloud Model Garden in the coming weeks and integrated into Google’s agent-driven programming tool, Antigravity; this move makes Google the first mainstream cloud provider to offer all three major closed-source large models—Gemini, Claude, and GPT.

Claude Mods: The first implementation of an attention interface, a human-programmable agent interaction layer

Claude Mods is Anthropic’s programmable plugin system for Claude Code, representing an early form of the “Attention Interface”—it shifts the traditional machine-oriented agent harness toward a customizable interaction layer centered on human attention. Users can write Mods in TypeScript/JS or have Claude generate them automatically using natural language, thereby determining when to be interrupted, what content to see, and what decisions to make.

Claude Mods: A programmable attention interface that reshapes the collaboration paradigm between humans and AI agents

Claude Mods is a programmable plugin system for developers launched by Anthropic, marking a shift of the ‘agent harness’ from a machine-side interface to a human-side interface—namely, the ‘Attention Interface’. It enables users to customize Claude Code’s behavior, rendering, and interaction logic via TypeScript/JavaScript, thereby facilitating co-design workflows between humans and AI. This mechanism has already been applied to built-in features (such as You Should Know) and supports users in building plugins themselves or having them generated with Claude’s assistance.

Reflection releases the open-source large model Beam: a 501B-parameter sparse MoE architecture that supports million-token contexts and multi-level inference control.

Reflection has released the open-source large language model Beam (501B parameters, with 23B effective activations per token), which employs a 256-expert sparse MoE architecture, supports 1M tokens of context, NVFP4/FP8 quantization, multi-level inference configuration, and on-premises deployment. All technical claims have not been independently verified, hardware requirements are high, and the community generally believes that individual users will find it difficult to afford.