Skip to content
AI NEWS JOURNAL
AI, IN PERSPECTIVE

Search AI news

Search headlines, summaries and articles to connect the dots.

Newest first

b12561

The update introduces a chunked GATED_DELTA_NET implementation for Vulkan, including shader additions, optimizations for RDNA4 and Ampere architectures, fixes for shared memory bank conflicts and NVIDIA shader compiler issues, and support for various subgroup sizes and backend configurations. It also enables the feature by default and adds diagnostic markers for device information.

b12489

A fix has been implemented to return a 400 error when both grammar and ignore_eos parameters are enabled in the server, preventing crashes and improving validation by moving the check into the sampler. Redundant tests have also been removed.

b12475

The Llama Chat UI has been upgraded to SvelteKit 3, integrating its built-in service worker for improved update detection and offline functionality. The migration includes updates to Vite 8, TypeScript 6, Storybook 10, and ESLint 10, along with the replacement of deprecated packages and resolution of npm audit issues through dependency overrides, including pinning minimatch to version 11.0.5 to address a security vulnerability. Changes also align with current Svelte patterns, such as using $state instead of stores and adopting new syntax for actions and class bindings.

b12364

A warning is now issued when the --spec-draft-p-min parameter is used with probabilistic draft sampling, as the draft's top probability does not reliably indicate how much of the draft the target model would accept under rejection sampling. The warning is skipped for drafters that do not use p_min, such as n-gram and DSpark.

b12377

The update refactors DFlash-based speculative decoding methods in llama.cpp by restructuring code organization, sharing components across models, splitting decoder graphs per head, and introducing dedicated classes for DFlash2 and LiliCorr drafts. It also adds support for new draft types, improves documentation, and removes unused code.

b12336

The update requires the position embedding tensor for HunyuanVL to be explicitly provided as F32 during model loading to prevent buffer overflow issues when using F64 tensors, ensuring compatibility and stability on first image requests.

v1.0.0

The llama.cpp project has released version 1.0.0, marking a significant milestone with numerous improvements across multiple platforms including CUDA, Vulkan, Metal, SYCL, OpenCL, and WebGPU. Key updates include enhanced support for quantized models (Q5_0, Q5_1, FP8), optimizations for flash attention and MoE caching, improved server security features like CORS protection and SSRF mitigation, UI enhancements such as sidebar reorganization and live-preview markdown editing, and better cross-platform compatibility. The release also includes bug fixes, CI/CD improvements, and updated dependencies.

b12189

The metal backend's kernel_group_norm_f32 was updated to correctly handle tensors with ne03 > 1 and non-contiguous inputs by dispatching one threadgroup per (group, i03) pair, computing per-batch statistics, clamping group element count for incomplete groups, and supporting strided access via separate strides for source and destination. The fix also restricts GROUP_NORM to f32 inputs and improves performance through specialized pipeline variants.

b12094

Optimized TOP_K performance on Hexagon backend by skipping bitonic sort for tiles whose maximum cannot surpass the k-th best value, replacing tile sort with top_k_scan() to reduce 256-float blocks using HVX max-reduce, and inserting only surviving elements into a sorted k-entry list. Supports k values from 2 to 64 and row widths over 1024 in both single-row and multi-row paths. NaN treated as -INF, ties preserve first-seen index. Eliminated scalar reads from VTCM in hex-top-k.

b11946

server : add realtime conversation and transcription on /v1/realtime (#31269) * server : add realtime conversation and transcription on /v1/realtime Co-authored-by: Xuan Son Nguyen Assisted-by: Claude Opus 5.5 * examples : default the realtime client to the server port 9931 Assisted-by: Claude Opus 5.5 * server : sample realtime responses right after their own decode Slots respond in turns on one context: sampling at the start of the next step read the logits of whichever slot decoded last, so concurrent sessions mixed their outputs. Assisted-by: Claude Opus 5.5 * move more stuff to server-realtime, making it self-contained * examples : fix the type check of the realtime client Assisted-by: Claude Opus 5.5 * examples : use the model card prompts in the realtime client Assisted-by: Claude O

Timeline and related coverage

b11922

tests : fix multi-seq split replay and partial-state cases for trained models (#31216) * tests : fix multi-seq split replay against trained models The rollback snapshot is captured by the graph that decoded the rolled-back tail, while the reference context decodes the same prefix with different batch shapes. The two states differ in the last bits and trained models amplify the difference with every replayed token. Compare only the first replayed token of each seq, then restore the rollback state into the reference context before the seq-1-only tail. Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * tests : require bit-exact seq-1 isolation in split replay The seq state save is not always bit-lossless - SWA-masked cells are not saved - so reload the rest

Timeline and related coverage

b11921

llama : remove obsolete n_pad from memory instances (#31200) Every caller passes n_pad = 1 and the only two users clamp it to 256. Drop the parameter, the members and the trivially true kv_size assert. Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11921/llama-b11921-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11921/llama-b11921-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11921/llama-b11921-xcfram

Timeline and related coverage

b11888

common, grammar: add recursive depth limit, make it a requirement (#31153) * common, grammar: add recursive depth limit, make it a requirement * catch more recursive functions * common : tidy up peg parser changes * grammar : reject candidates in a loop, without a depth limit The depth limit in llama_grammar_reject_candidates_for_stack counted the code points of a vocab token, so any token of 256 chars or more was always rejected under a grammar, such as the 512-space token of GLM-4.7-Flash or the long space tokens of Laguna-S-2.1. The rejection now walks the code points with its own stack of levels instead of recursing: there is no limit and no stack risk, whatever the length of a token from the GGUF, and the masks match master. * grammar : track rejected candidates as indices into the in

Timeline and related coverage