Skip to content
AI NEWS JOURNAL
AI, IN PERSPECTIVE

Search AI news

Search headlines, summaries and articles to connect the dots.

Newest first

b12561

The update introduces a chunked GATED_DELTA_NET implementation for Vulkan, including shader additions, optimizations for RDNA4 and Ampere architectures, fixes for shared memory bank conflicts and NVIDIA shader compiler issues, and support for various subgroup sizes and backend configurations. It also enables the feature by default and adds diagnostic markers for device information.

b12489

A fix has been implemented to return a 400 error when both grammar and ignore_eos parameters are enabled in the server, preventing crashes and improving validation by moving the check into the sampler. Redundant tests have also been removed.

b12475

The Llama Chat UI has been upgraded to SvelteKit 3, integrating its built-in service worker for improved update detection and offline functionality. The migration includes updates to Vite 8, TypeScript 6, Storybook 10, and ESLint 10, along with the replacement of deprecated packages and resolution of npm audit issues through dependency overrides, including pinning minimatch to version 11.0.5 to address a security vulnerability. Changes also align with current Svelte patterns, such as using $state instead of stores and adopting new syntax for actions and class bindings.

PyTorch Foundation Ambassador Community Highlights November 2026

PyTorch Foundation Ambassadors bring people together to learn, share technical knowledge and contribute to PyTorch Foundation projects and the wider open-source AI ecosystem. In this November roundup, we highlight some... The post PyTorch Foundation Ambassador Community Highlights November 2026 appeared first on PyTorch.

Does the Ranking Hold? A Practitioner-Facing Audit of Hallucination Detectors on Small, Quantized, On-Device LLMs

arXiv:2611.17339v1 Announce Type: new Abstract: Hallucination detection for large language models is dominated by methods evaluated on 7B-70B research models with abundant sampling budgets and exact-match or ROUGE correctness labels. The small models practitioners actually ship on-device, Phi-3-mini, Llama-3.2-3B-Instruct, Qwen2.5-3B-Instruct, and Gemma-2-2B-Instruct, typically at 4-bit precision, sit outside this evidence base. We present a reproducible one-GPU audit of four detectors spanning three paradigms: length-normalized token-margin calibration, a lexical n-gram baseline, and two sampling-based consistency methods (semantic entropy and SelfCheckGPT-NLI) on two small open instruction-tuned models (Llama-3.2-3B-Instruct and Qwen2.5-3B-Instruct) loaded at 4-bit NF4 quantization on a single Colab T4 GPU, evaluated on closed-book TriviaQA. Cross-model ranking transfer is poor: Spearman rho is -0.80 and Kendall tau-b is -0.667, meaning the detector ranking actively inverts between the two models, and the cheap lexical baseline is statistically indistinguishable from the sampling-based consistency detectors on both models under bootstrap 95 percent confidence intervals. As a second, methodological contribution, we show that the standard token-overlap-to-multiple-choice correctness labeler for TruthfulQA-Generation produces near-chance AUROC (0.46-0.59) for all four detectors on both models, a labeler failure, not a detector failure, since the same detectors reach 0.69-0.89 AUROC on TriviaQA under the identical pipeline. We release the harness, prompts, and per-item scores and frame the detector choice explicitly as a fairness-of-error-rates allocation problem for practitioners deploying on resource-constrained, on-device hardware.

b12364

A warning is now issued when the --spec-draft-p-min parameter is used with probabilistic draft sampling, as the draft's top probability does not reliably indicate how much of the draft the target model would accept under rejection sampling. The warning is skipped for drafters that do not use p_min, such as n-gram and DSpark.

b12377

The update refactors DFlash-based speculative decoding methods in llama.cpp by restructuring code organization, sharing components across models, splitting decoder graphs per head, and introducing dedicated classes for DFlash2 and LiliCorr drafts. It also adds support for new draft types, improves documentation, and removes unused code.

b12336

The update requires the position embedding tensor for HunyuanVL to be explicitly provided as F32 during model loading to prevent buffer overflow issues when using F64 tensors, ensuring compatibility and stability on first image requests.

Fast by Default: Compile-Time Heuristics for Helion Kernels

TL;DR Helion now uses compile-time heuristics that analyze a kernel’s IR and dataflow to pick fast settings without slow autotuning. It classifies the kernel as matmul, reduction, or pointwise and... The post Fast by Default: Compile-Time Heuristics for Helion Kernels appeared first on PyTorch.