Search headlines, summaries and articles to connect the dots.
The latest update to llama.cpp introduces reference-counted checkpoints instead of deep-copies, reducing memory overhead and improving efficiency. This change adapts the slot save/restore checkpoint mechanism to shared_ptr checkpoint lists and common_state_data buffers, with contributions from multiple developers and AI assistants.
The update enables f32 and q5_k multi-column GEMV by default for Adreno X2 on X2E devices, improving performance for specific computational tasks in the llama.cpp framework.
server : add realtime conversation and transcription on /v1/realtime (#31269) * server : add realtime conversation and transcription on /v1/realtime Co-authored-by: Xuan Son Nguyen Assisted-by: Claude Opus 5.5 * examples : default the realtime client to the server port 9931 Assisted-by: Claude Opus 5.5 * server : sample realtime responses right after their own decode Slots respond in turns on one context: sampling at the start of the next step read the logits of whichever slot decoded last, so concurrent sessions mixed their outputs. Assisted-by: Claude Opus 5.5 * move more stuff to server-realtime, making it self-contained * examples : fix the type check of the realtime client Assisted-by: Claude Opus 5.5 * examples : use the model card prompts in the realtime client Assisted-by: Claude O
model : add LiquidAI/d1-2B and LiquidAI/d1-omni-2B (#31275) * model : add LiquidAI/d1-2B and LiquidAI/d1-omni-2B Assisted-by: Claude Opus 5.5 * convert : keep the speech task templates out of the d1 GGUFs Assisted-by: Claude Opus 5.5 * small refactoring: no_batching --------- Co-authored-by: Xuan Son Nguyen **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11945/llama-b11945-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11945/llama-b11945-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b119
model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B (#31257) * model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B Co-authored-by: Xuan Son Nguyen Assisted-by: Claude Opus 5.5 * convert : read the LFM3-Audio config.json as released A transformers that knows lfm3_audio rewrites the config into another layout (no lfm/encoder keys). Assisted-by: Claude Opus 5.5 * mtmd : disable flash attention for the LFM3-Audio depthformer Assisted-by: Claude Opus 5.5 * tts : add a voice option for LFM3-Audio and LFM3-Omni The model card offers a female and a male voice; --tts-voice for llama-tts, "voice" for the server /tts. Assisted-by: Claude Opus 5.5 * clean up template handling * convert : detect LFM3-Audio and LFM3-Omni by model_type Assisted-by: Claude Opus 5.5 * move banned tokens -
common : support deferred tools and tool_reference content parts (#31256) * common : support deferred tools and tool_reference content parts Assisted-by: Claude Opus 5.5 * common : credit the deferred tools co-author Co-authored-by: Aldehir Rojas <765227+aldehir@users.noreply.github.com> Assisted-by: Claude Opus 5.5 * tests : fix the type check of the deferred tools test Assisted-by: Claude Opus 5.5 **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11940/llama-b11940-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11940/llama-b11940-bin-maco
model : add LiquidAI/LFM3-1.7B and LiquidAI/LFM3-VL-2B (#31252) * model : add LiquidAI/LFM3-1.7B and LiquidAI/LFM3-VL-2B Co-authored-by: Sigbjørn Skjæret Co-authored-by: Xuan-Son Nguyen Assisted-by: Claude Opus 5.5 * tests : set the SWA rope base of the LFM3 test model Assisted-by: Claude Opus 5.5 * llama : enable recurrent state rollback for LFM3 Assisted-by: Claude Opus 5.5 * Apply batched suggestions from code review Co-authored-by: Sigbjørn Skjæret Co-authored-by: Tarek Dakhran * gguf-py : align the lfm3vl mapping comments Assisted-by: Claude Opus 5.5 * llama : fix CRLF line endings from the review suggestions Assisted-by: Claude Opus 5.5 * tests : add the LFM3 Metal fusion baseline Assisted-by: Claude Opus 5.5 * llama : keep the LFM3-VL depthwise conv kernels in F32 when quantizing As
llama-quant : fix 4D tensor quantization using ggml_nrows() and name-based expert detection (#30820) * llama-quant : fix quantizing 4D tensors nrows_total was ne[1]*ne[2], so for a 4D tensor only the first ne[2] slices were quantized and the rest of the rows were dropped. Use ggml_nrows(tensor) to count all rows, including ne[3]. Fixes #29916 * llama-quant : detect expert tensors by name and fix imatrix size for 4D tensors The MXFP4_MOE path used ne[2] > 1 to detect MoE expert tensors. This is wrong for non-expert tensors with 3 or more dims, e.g. MLA projections (BailingMoE3) and 4D conv kernels. Use the tensor name instead: all tensors used by GGML_OP_MUL_MAT_ID are named *_exps or *_chexps. This also removes the BailingMoE3 special case. The imatrix holds one vector of ne[0] values per
