Skip to content
AI NEWS JOURNAL
AI, IN PERSPECTIVE

Search AI news

Search headlines, summaries and articles to connect the dots.

Newest first

180B Parameters, 32GB of RAM: Pushing MoE Inference to the Limit with Transformers

The author explores streaming MoE (Mixture of Experts) model experts directly from SSD to achieve near-Strata performance on a 32GB MacBook M5, using a custom implementation in Hugging Face Transformers. By optimizing expert caching, leveraging Metal kernels, and minimizing host-GPU synchronization delays, the approach reaches ~14 tok/s—within 6% of Strata's performance—demonstrating that efficient expert streaming is feasible on consumer hardware without C++ extensions.

Time-Series Models in the Streamhouse: A New Frontier and Use Cases

IBM's time-series foundation models, with over 50 million downloads, are now natively integrated into Confluent's platform, enabling real-time forecasting, anomaly detection, and optimization on streaming data using SQL. These models run on CPUs in both cloud and on-premises environments, supporting mission-critical applications across manufacturing, retail, banking, and airlines by delivering timely insights closer to decision-making points. The integration leverages Apache Flink and Confluent’s Streamhouse architecture to process data streams, apply AI-driven intelligence, and enable actions such as production optimization, fraud detection, supply chain planning, and trading execution—all in real time.

Small Open-Weight Language Models for Crowd Decision-Making: Capabilities and Limits of Output-Level Fusion and Linguistic Aggregation

arXiv:2611.17589v1 Announce Type: new Abstract: Small open-weight language models can run crowd decision-making (CDM) on an institution's own infrastructure, which keeps the checkpoint, the inference settings and the opinions under its control. Their judgements vary across repeated queries; whether repeating and aggregating them, often through the 2-tuple linguistic representation, settles the decision rather than the labels is untested. Five open-weight models of 8-36B total parameters on one consumer GPU and a hosted reference classify 474 TripAdvisor reviews of four London restaurants ten times each; three open-weight models repeat the protocol on 600 reviews of twelve Madrid restaurants. Continuous evaluations preserve resolution: snapping each review leaves 5 distinct values against 21-65, and snapping at every level at most two collective values. Sensitivity to the participants persists once run-to-run variability is reduced: in London, redrawing the reviewers keeps the winner on 47-81% of replicas against 86-100% for redrawing the runs. With simultaneous intervals over every pair of alternatives under crowd resampling, no system on either corpus names a single winner: in London every system keeps all four restaurants as candidates, and in Madrid the candidate sets hold five to eight of the twelve. Extra runs and output-level fusion stabilize the winner but do not remove that sensitivity, and the fusion does not beat its best member. The three larger open-weight models name the hosted reference's London winner and at least match its per-review correlation with the reviewers' stars, so small open-weight models can support local CDM, reported as candidate sets with margins rather than forced rankings.

XYBench: Helping with the problem behind the question

XYBench, a benchmark of 8,115 real-world questions spanning programming and everyday tasks, tests AI assistants' ability to recognize and respond to users' misconceptions—such as asking how to thicken frosting when the real issue is warm cupcakes. Developed with researchers from Carnegie Mellon, NVIDIA, and Johns Hopkins, XYBench measures whether models identify underlying goals (X), address them effectively, and steer users away from flawed approaches (Y). Evaluations of models like Claude Sonnet 4.6, GPT-5, Gemini 3.1 Pro, Qwen3.5-9B, and DeepSeek-V4 Pro showed they frequently focused on the stated request rather than the actual goal, with limited success in misconception identification. Human reference answers outperformed all models, highlighting a gap in pragmatic reasoning. The benchmark uses automated and human evaluation to assess pragmatic solution, emphasis, and misconception identification, offering a tool for researchers to improve AI assistants’ ability to provide helpful advice even when users ask the wrong question.

LLMON: An LLM-Native Markup Language, now in Granite 4.1 3B and 8B

IBM has introduced LLMON (LLM Object Notation), a new markup language designed to explicitly communicate structure and semantics to large language models. Built to address issues like prompt injection attacks and brittle behavior in smaller models, LLMON uses reserved special-token delimiters to mark semantically coherent components such as instructions, data, and requirements, enabling precise binding between them. Two experimental preview models—granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview—are now available, alongside a GitHub repository with conversion tools, workflows, and examples. LLMON enhances robustness by separating instruction from data, reducing indirect prompt injection success rates nearly to zero while maintaining performance comparable to standard Granite 4.1 Instruct models. It also supports custom schemas via LoRA fine-tuning, as demonstrated in ScienceWorld and tool-calling agent use cases. The approach is distinct from JSON, which lacks model-friendly structure, and complements existing chat templates by adding semantic clarity across roles.

LLMs and the Homogenization of Teaching

arXiv:2611.14828v1 Announce Type: new Abstract: Large language models (LLMs) are known to lack diversity in open-ended text generation, a limitation that is especially consequential in education, where homogeneity can narrow the kinds of thinking and disciplinary work students are invited to do. We study diversity in two educational generation tasks: math items and essay feedback. We compare frontier and open-weight models with human-authored reference distributions using lexical, semantic, and domain-grounded measures. For math, we examine the mathematical activities students are asked to perform; for feedback, we examine the revision work requested and the rhetorical strategies used to communicate it. We evaluate direct prompting, two diversity prompting methods, and post-training. Across both tasks, directly prompted model outputs are markedly less diverse than human-authored content--78% higher on lexical (Self-BLEU) and 51% higher on semantic (embedding) similarity--and represent a narrower range of instructional forms. Diversity prompting and post-training increase variation largely through surface features without broadening the instructional repertoire. Models inherit and amplify the most normative routines of human instruction. Overcoming mode collapse in education will require richer instructional data and investment in teachers' creative capacity and professional judgment.

Nemotron-Personas-Finance-Japan: A Synthetic Dataset Capturing Japan's Demographics and Financial Context

NVIDIA and Elith have released Nemotron-Personas-Finance-Japan, a large-scale open-source synthetic dataset featuring 9 million personas based on Japan’s official demographic, labor, and financial statistics. Built using NVIDIA’s NeMo Data Designer and SDG-PGMs framework, the dataset includes 50 fields per record with natural Japanese text, covering demographics, household finances, occupations, financial literacy, and personality traits. Designed for commercial and non-commercial use under CC BY 4.0, it enables privacy-safe development of financial AI applications such as conversational agents, customer simulations, and bias testing, while aligning with Japan’s cultural and regulatory context.

Your Experts Have Partners. There's HOPE for Keeping Them Together

HOPE, a second-order expert pruning method for Mixture-of-Experts (MoE) language models, accounts for pairwise expert interactions by optimizing a quadratic program based on an interaction matrix derived from calibration data. Unlike first-order methods that prune experts independently, HOPE preserves cooperative expert groups, leading to better performance—especially at high pruning rates like 50%. On benchmarks such as SWE-Bench Pro, HOPE outperforms state-of-the-art first-order methods like REAP by up to +6.1%, maintaining model quality while significantly reducing memory footprint. The method requires only a single calibration pass and efficient quadratic programming, with negligible overhead. It has been validated on Qwen3.5 and GLM-4.5 models, showing consistent improvements across diverse tasks and calibration sets.

Why Pay for Long Reasoning When Rewrite Do Trick? Causal Effects of Reasoning History Compaction on Future Generation and Agent Cost

arXiv:2611.03852v1 Announce Type: new Abstract: Reasoning substantially improves language models' ability to solve difficult problems, but long reasoning traces make these gains expensive. Efficient reasoning should express useful analysis in fewer tokens by reducing repetition and text that adds little value to solving the problem. Progress in proprietary models motivates this goal, yet their hidden scratchpads limit direct study of how it is achieved. We investigate whether open-weight models can generate less verbose reasoning by changing the form of their own earlier thoughts, without retraining. Our intervention rewrites prior reasoning into compact state records before pending tool results are revealed, then compares continuations from original and compact histories. Across the tested models, accepted compact histories usually shorten future reasoning; a coding case study also shows less repeated verification after a shared candidate solution. In software-agent deployments, compaction can reduce total spending, although task quality and rewrite-inclusive time depend on the target and compressor. These findings identify reasoning history as one way to reduce future reasoning-token use and motivate model-specific compaction policies evaluated for both efficiency and retained capability.

$\Psi$-0.6: Adaptive Streams of Thought in a Unified Vision-Language World Model

arXiv:2610.32177v1 Announce Type: new Abstract: Humans reason about the physical world by flexibly combining different forms of inference as a problem unfolds. Achieving this flexibility is a major open challenge in AI: capabilities often remain separated across specialized models, limiting how they can be composed and jointly adapted to a task. We introduce $\Psi$-0.6, a unified vision-language world model that adaptively composes perception, language, geometry, and simulation within a shared reasoning process we call streams of thought. These streams interleave images, text, and explicit representations of geometry and motion, all of which the model can generate and interpret through a shared token interface. $\Psi$-0.6 surpasses leading open-weight models zero-shot on multi-view spatial

Timeline and related coverage

Can Test-Time Memory Evolution Outperform Test-Time Training for Scientific Discovery?

arXiv:2610.32046v1 Announce Type: new Abstract: Recent \emph{test-time training} (TTT) frameworks have enabled open-weight large language models (LLMs) to outperform inference-only frameworks powered by frontier models on scientific discovery tasks. However, TTT introduces additional computational cost for online weight updates during the discovery process, making computational efficiency an important consideration when scaling to larger models. This motivates us to explore an efficient \emph{test-time adaptation} method for scientific discovery. We introduce \emph{Test-Time Memory Evolution} (TTME), a plug-and-play test-time adaptation method, as an alternative to TTT. TTME continually updates the external memory by distilling the experience to guide subsequent discovery without updating

Can Test-Time Memory Evolution Outperform Test-Time Training for Scientific Discovery?
Timeline and related coverage