Search headlines, summaries and articles to connect the dots.
Optimized TOP_K performance on Hexagon backend by skipping bitonic sort for tiles whose maximum cannot surpass the k-th best value, replacing tile sort with top_k_scan() to reduce 256-float blocks using HVX max-reduce, and inserting only surviving elements into a sorted k-entry list. Supports k values from 2 to 64 and row widths over 1024 in both single-row and multi-row paths. NaN treated as -INF, ties preserve first-seen index. Eliminated scalar reads from VTCM in hex-top-k.
The update introduces the TOP_K_MASK operation for Hexagon, simplifying the kernel and eliminating redundant host configuration.
The llama.cpp release b12098 includes accurate support for Qwen3-VL video prompt format and provides precompiled binaries for multiple platforms including macOS, Linux, Windows, Android, and openEuler, with various backend options such as CUDA, Vulkan, ROCm, OpenVINO, SYCL, and Snapdragon-specific optimizations. The release also features a standalone UI package and detailed setup guides for specific hardware configurations.
The update introduces initial support for the LIGHTNING_INDEXER operation in the Hexagon backend, including optimizations for HMX prefill, q4_0 and f32 K support, pipelined HMX paths, improved DMA pipeline with canonical double buffering, and enhanced kernel parameter alignment. It also adds options to force HMX/HVX execution paths and supports dimension collapse.
The update resolves issues with -hf tag matching in longer quantization names, ensuring tags start at token boundaries. It also adds support for underscores as separators in quantization tags, improving compatibility with filenames like step3p5_flash_Q4_K_S-00001-of-00012.gguf.
The gguf writer now writes host-buffer tensor data directly to the file, skipping the temporary staging vector, which eliminates per-tensor allocation, zero-initialization, and memcpy operations.
The ggml library now checks view bounds based on the actual bytes reached by strides, improving accuracy for strided views. Views are required to start from contiguous elements along dimension 0, ensuring correct data access. Convolution windows in delta_net are created with all rows at once to prevent stride-related issues.
The llama.cpp release b12109 is now available, offering pre-compiled binaries for macOS, Linux, Windows, Android, and openEuler across various architectures and backends including CPU, CUDA, Vulkan, ROCm, OpenVINO, SYCL, and specialized hardware like Snapdragon. The release includes support for Apple Silicon and Intel macOS, multiple Linux distributions, and detailed setup guides for specific platforms.
The llama.cpp project has been updated to version 0.9.0, with new releases available for multiple platforms including macOS, Linux, Windows, Android, and iOS. The update includes support for various backends such as CUDA, Vulkan, ROCm, OpenVINO, SYCL, and Snapdragon-specific hardware. Some builds are marked as disabled or experimental, and detailed setup guides are provided for specific configurations.
ComfyUI v0.44.0 introduces multiple new partner nodes for tools like Patina, VEED, BFL, Ideogram, Pruna, MiniMax, Topaz, Sonilo, and OpenAI, adding features such as higher resolution video upscaling, context window editing, and new models. Enhancements include support for ProRes and custom chroma subsampling in video saving, improved frontend handling with pinned versions, optimized tensor comparisons, and better VAE compute dtype control. Performance optimizations address redundant concatenations and memory management, while bug fixes resolve crashes related to VAE decoding and prefetch queue cleanup. The release also updates workflow templates, embedded documentation, and default settings, with contributions from new developers.
Dify 1.19.0 enhances observability with nested workflow tracing, allowing users to follow execution steps inside Workflow Tools, inspect LLM calls, and maintain context across nested invocations. The release also enables portable app packages via App packages, improves Agent file uploads and previews, introduces Skill Hub automation through CLI commands, and upgrades the bundled plugin daemon to version 0.6.14-local. Key changes include support for format v2 app packages, enhanced security, improved editing workflows, and updated environment variables and API endpoints.
tests : adjust constants to reduce irrelevant warnings (#31254) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11954/llama-b11954-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11954/llama-b11954-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11954/llama-b11954-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11954/llama-b11954-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b1
cuda : add top-k MoE fusion for 384 experts (#31288) Assisted-by: Codex **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11953/llama-b11953-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11953/llama-b11953-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11953/llama-b11953-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11953/llama-b11953-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/dow
chat : stop qwen3-coder reasoning and content before template separators (#30557) * chat : strip trailing whitespace from parsed reasoning and content Assisted-by: Claude Fable 5.1 * chat : stop qwen3-coder reasoning and content before template separators Assisted-by: Claude * tests : remove the Qwen parser case from the chat session tests Assisted-by: Claude * common : handle partial prefix until match at end of generation --------- Co-authored-by: Zeshawn Zahid Co-authored-by: Alde Rojas **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11951/llama-b11951-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel
server : add realtime conversation and transcription on /v1/realtime (#31269) * server : add realtime conversation and transcription on /v1/realtime Co-authored-by: Xuan Son Nguyen Assisted-by: Claude Opus 5.5 * examples : default the realtime client to the server port 9931 Assisted-by: Claude Opus 5.5 * server : sample realtime responses right after their own decode Slots respond in turns on one context: sampling at the start of the next step read the logits of whichever slot decoded last, so concurrent sessions mixed their outputs. Assisted-by: Claude Opus 5.5 * move more stuff to server-realtime, making it self-contained * examples : fix the type check of the realtime client Assisted-by: Claude Opus 5.5 * examples : use the model card prompts in the realtime client Assisted-by: Claude O
model : add LiquidAI/d1-2B and LiquidAI/d1-omni-2B (#31275) * model : add LiquidAI/d1-2B and LiquidAI/d1-omni-2B Assisted-by: Claude Opus 5.5 * convert : keep the speech task templates out of the d1 GGUFs Assisted-by: Claude Opus 5.5 * small refactoring: no_batching --------- Co-authored-by: Xuan Son Nguyen **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11945/llama-b11945-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11945/llama-b11945-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b119
server: expose the reasoning effort levels of the chat template (#31271) * server: expose the reasoning effort levels of the chat template The capability probe renders a user turn for each candidate level (minimal, low, medium, high, xhigh, max) and for an unknown level. A level is honored when it renders without raising and differently from the unknown level, which shows the template fallback. Levels sharing a render are aliases and the highest one is kept. The caps serialize to JSON, and chat_template_caps in /props reports the list as reasoning_effort_levels, from lowest to highest. * jinja: keep the caps free of the json dependency The caps keep reporting their flags as a map, and the effort levels stay a plain member. common_chat_templates_get_caps() builds the JSON the server reports
model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B (#31257) * model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B Co-authored-by: Xuan Son Nguyen Assisted-by: Claude Opus 5.5 * convert : read the LFM3-Audio config.json as released A transformers that knows lfm3_audio rewrites the config into another layout (no lfm/encoder keys). Assisted-by: Claude Opus 5.5 * mtmd : disable flash attention for the LFM3-Audio depthformer Assisted-by: Claude Opus 5.5 * tts : add a voice option for LFM3-Audio and LFM3-Omni The model card offers a female and a male voice; --tts-voice for llama-tts, "voice" for the server /tts. Assisted-by: Claude Opus 5.5 * clean up template handling * convert : detect LFM3-Audio and LFM3-Omni by model_type Assisted-by: Claude Opus 5.5 * move banned tokens -
common : support deferred tools and tool_reference content parts (#31256) * common : support deferred tools and tool_reference content parts Assisted-by: Claude Opus 5.5 * common : credit the deferred tools co-author Co-authored-by: Aldehir Rojas <765227+aldehir@users.noreply.github.com> Assisted-by: Claude Opus 5.5 * tests : fix the type check of the deferred tools test Assisted-by: Claude Opus 5.5 **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11940/llama-b11940-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11940/llama-b11940-bin-maco
model : add LiquidAI/LFM3-1.7B and LiquidAI/LFM3-VL-2B (#31252) * model : add LiquidAI/LFM3-1.7B and LiquidAI/LFM3-VL-2B Co-authored-by: Sigbjørn Skjæret Co-authored-by: Xuan-Son Nguyen Assisted-by: Claude Opus 5.5 * tests : set the SWA rope base of the LFM3 test model Assisted-by: Claude Opus 5.5 * llama : enable recurrent state rollback for LFM3 Assisted-by: Claude Opus 5.5 * Apply batched suggestions from code review Co-authored-by: Sigbjørn Skjæret Co-authored-by: Tarek Dakhran * gguf-py : align the lfm3vl mapping comments Assisted-by: Claude Opus 5.5 * llama : fix CRLF line endings from the review suggestions Assisted-by: Claude Opus 5.5 * tests : add the LFM3 Metal fusion baseline Assisted-by: Claude Opus 5.5 * llama : keep the LFM3-VL depthwise conv kernels in F32 when quantizing As
