Introducing Control Room: Human control over AI workloads
Learn how OpenAI’s Control Room tracks model workloads across our research infrastructure and can rapidly stop them when safety or security concerns arise.
AI NEWS JOURNAL
Search AI news
Search headlines, summaries and articles to connect the dots.
PINNeval: A Comprehensive Evaluation Standard for Physics-Informed Neural Networks
arXiv:2611.08559v1 Announce Type: new Abstract: The Physics-Informed Neural Network (PINN) literature has produced a rapidly expanding catalogue of methods. We divide a PINN training pipeline systematically into four components and observe that many existing benchmarks study these components in isolation. These components are domain-importance adaptation, loss balancing, neural network architecture, and optimization scheme. Moreover, the studies set the surrounding and not considered components of the pipeline to a minimal default that is no longer representative of current best practice approaches. Conclusions drawn under such conditions may not transfer to competitive pipelines. In this work, we propose a new evaluation protocol that (1) treats the four components as an integrated system; (2) crucially requires competitive implementations of the components not under investigation; (3) specifies the metrics and reporting format; and (4) is accompanied by structured documentation that ensures the traceability of evaluations across studies. Based on this comprehensive protocol, we release PINNpipe, an innovative JAX package containing 18 different methods from the current literature as well as nine efficiently implemented benchmark problems. Using PINNpipe, we illustrate our evaluation protocol in relation to established benchmark studies and report results that challenge common assumptions.
Say Less, Mean Better: Training-Free Verbosity Control for Language Models
arXiv:2611.04107v1 Announce Type: new Abstract: Large language models (LLMs) often generate responses longer than necessary, increasing inference latency and computational cost. While prompting can encourage conciseness, controlling output length without compromising response completeness remains challenging. Aggressive early stopping can reduce generation costs but risks terminating responses prematurely. We introduce MinimalLM, a training-free framework for reliable verbosity control that combines uncertainty-aware, length-dependent stop-token adjustment with structural completeness guardrails. MinimalLM separates the decision of when to stop from whether stopping is structurally appropriate, coordinating response-level instructions, decoding-time intervention, and completion checks without modifying model parameters or requiring auxiliary models. We evaluate MinimalLM across three open-source LLMs and 330 questions, producing 5,940 responses across baseline and ablation configurations. MinimalLM reduces generated tokens by 47.6%, accelerates individual responses by 19-68%, and eliminates structurally truncated outputs in the primary evaluation. Despite substantial token savings, it achieves an average quality score of 8.04/10, compared with 8.42/10 for unmodified models and 6.46/10 for length-matched truncation. Evaluations across diverse subject domains (such as healthcare, programming, law, and science) demonstrate robust efficiency gains, while ablations establish the complementary roles of decoding control and completeness constraints. Our findings highlight that efficient generation requires more than producing fewer tokens: it requires controlling when generation ends while preserving response integrity. MinimalLM offers a lightweight, training-free approach toward this goal.
Kimi releases K3.1 Pro, offering more features at a lower price to challenge Opus 5.5.
Author | Yu Hang Yuan Editor | Jing Yu On the morning of October 23, Kimi officially posted a 24-second video on X and its Video Account. The scene shows a retro‑style vintage computer, with Kimi Code running on the screen and lines of Three.js code being typed into the terminal, generating a complex 3D scene. Keen-eyed netizens quickly spotted the answer in the model bar at the top left: “Model: Kimi K3.1.” Kimi’s official teaser video for K3.1 | Image source: X @Kimi_Moonshot Using an antiquated machine to run some of today’s most cutting‑edge 3D code—this stark contrast itself conveys what Moonshot AI is trying to say. A dozen hours later, just after midnight on October 24, right on Programmer’s Day, Kimi K3.1 Pro officially launched, accompanied by a Highspeed version that’s two to three times faster. The company promises that the full model weights will be released before November 4. More than three months have passed since the previous generation, K3, was unveiled. In those three months, Moonshot AI has experienced more than most companies see in an entire year. 01 “More for the same price”—what’s new in 0.1? First, let’s clarify what K3.1 Pro is. It isn’t an entirely new model. Moonshot AI explicitly states that K3.1 Pro is a post‑training variant of K3; the underlying architecture remains unchanged—still a hybrid expert architecture with 2.8 trillion total parameters and 104 billion activation parameters, still featuring a 1 million‑token context window, and still natively supporting image and video inputs. What has changed is the post‑training phase. According to the company, they’ve invested “significantly increased post‑training compute power,” conducting large‑scale joint reinforcement learning across hundreds of downstream tasks. To support this, they completely reengineered the underlying infrastructure—including the reinforcement learning framework, sandbox environment, and long‑context reasoning capabilities. In essence, the same brain has been retrained from scratch on how to perform its tasks. The results are reflected in benchmark scores. On Terminal‑Bench 4.0, which evaluates coding agents, K3.1 Pro achieved 65.7%, surpassing GPT‑6 Astra’s 58.2% and falling short of Claude Opus’s 66.4% by less than one percentage point. As for the previous generation, K3…
Tampered ML: Evaluating LLM Judgment of Machine Learning Research
arXiv:2610.32114v1 Announce Type: new Abstract: One approach to accelerating progress on AI alignment is to automate parts of alignment research itself. Doing so requires systems that can perform reliably on hard-to-supervise research tasks, including identifying flaws in research proposals and evaluating whether a proposed experiment supports its intended claim. To evaluate this capability, we introduce Tampered ML, a benchmark of 1,858 item-pairs, each consisting of an experiment proposal reconstructed from a published machine learning paper and a minimally edited counterfactual that introduces a targeted methodological flaw. Models must identify the more methodologically sound proposal in both presentation orders. A human validation study on a stratified sample estimates that 86% of ite
NVIDIA releases Kumo Relational: the first foundation model for structured data that outperforms supervised learning without any training.
NVIDIA has launched Kumo Relational, a pre-trained relational foundation model that can perform predictive tasks—such as churn prediction, fraud detection, and demand forecasting—directly on enterprise‑grade relational databases, without the need for task fine-tuning, feature engineering, or table flattening. In 41 benchmark tests spanning 15 databases, it outperformed supervised models like XGBoost across the board, and on the SAP SALT enterprise benchmark, its MRR metric surpassed AutoGluon by 8%. The model supports databases with up to 500 billion rows and enables zero‑code predictive queries via Predictive Query Language (PQL).
Extending the Context Length to 1M Tokens!
API Documentation (Chinese) HuggingFace Demo ModelScope Demo Introduction After the release of Qwen2.5, we heard the community’s demand for processing longer contexts. In recent months, we have made many optimizations for the model capabilities and inference performance of extremely long context. Today, we are proud to introduce the new Qwen2.5-Turbo version, which features: Longer Context Support: We have extended the model’s context length from 128k to 1M, which is approximately 1 million English words or 1.
Introducing Qwen2-Math
GITHUB HUGGING FACE MODELSCOPE DISCORD 🚨 This model mainly supports English. We will release bilingual (English and Chinese) math models soon. Introduction Over the past year, we have dedicated significant effort to researching and enhancing the reasoning capabilities of large language models, with a particular focus on their ability to solve arithmetic and mathematical problems. Today, we are delighted to introduce a series of math-specific large language models of our Qwen2 series, Qwen2-Math and Qwen2-Math-Instruct-1.
Hello Qwen2
GITHUB HUGGING FACE MODELSCOPE DEMO DISCORD Introduction After months of efforts, we are pleased to announce the evolution from Qwen1.5 to Qwen2. This time, we bring to you: Pretrained and instruction-tuned models of 5 sizes, including Qwen2-0.5B, Qwen2-1.5B, Qwen2-7B, Qwen2-57B-A14B, and Qwen2-72B; Having been trained on data in 27 additional languages besides English and Chinese; State-of-the-art performance in a large number of benchmark evaluations; Significantly improved performance in coding and mathematics; Extended context length support up to 128K tokens with Qwen2-7B-Instruct and Qwen2-72B-Instruct.