b12125
The update enables f32 and q5_k multi-column GEMV by default for Adreno X2 on X2E devices, improving performance for specific computational tasks in the llama.cpp framework.
Search headlines, summaries and articles to connect the dots.
The update enables f32 and q5_k multi-column GEMV by default for Adreno X2 on X2E devices, improving performance for specific computational tasks in the llama.cpp framework.
The fix addresses intermittent CI failures in TOPK_MOE tests caused by near ties in logits, particularly with uniform random inputs and narrow-range gating functions like sqrt_softplus. The solution involves redrawing logits until group selection and top-k decisions have a relative margin of 1e-4 in double precision, and constructing logits by inverting the gating function using a grid-based approach to ensure consistent expert selection across backends.
As artificial intelligence and accelerated computing continue to push the boundaries of what's possible, the latest NVIDIA CUDA release arrives to unlock...
Article URL: https://fireflyk64.github.io/wcmulti/ Comments URL: https://news.ycombinator.com/item?id=50369895 Points: 1 # Comments: 1
Optimized TOP_K performance on Hexagon backend by skipping bitonic sort for tiles whose maximum cannot surpass the k-th best value, replacing tile sort with top_k_scan() to reduce 256-float blocks using HVX max-reduce, and inserting only surviving elements into a sorted k-entry list. Supports k values from 2 to 64 and row widths over 1024 in both single-row and multi-row paths. NaN treated as -INF, ties preserve first-seen index. Eliminated scalar reads from VTCM in hex-top-k.
The update introduces the TOP_K_MASK operation for Hexagon, simplifying the kernel and eliminating redundant host configuration.
The llama.cpp release b12098 includes accurate support for Qwen3-VL video prompt format and provides precompiled binaries for multiple platforms including macOS, Linux, Windows, Android, and openEuler, with various backend options such as CUDA, Vulkan, ROCm, OpenVINO, SYCL, and Snapdragon-specific optimizations. The release also features a standalone UI package and detailed setup guides for specific hardware configurations.
The update introduces initial support for the LIGHTNING_INDEXER operation in the Hexagon backend, including optimizations for HMX prefill, q4_0 and f32 K support, pipelined HMX paths, improved DMA pipeline with canonical double buffering, and enhanced kernel parameter alignment. It also adds options to force HMX/HVX execution paths and supports dimension collapse.
The update resolves issues with -hf tag matching in longer quantization names, ensuring tags start at token boundaries. It also adds support for underscores as separators in quantization tags, improving compatibility with filenames like step3p5_flash_Q4_K_S-00001-of-00012.gguf.
The gguf writer now writes host-buffer tensor data directly to the file, skipping the temporary staging vector, which eliminates per-tensor allocation, zero-initialization, and memcpy operations.
The ggml library now checks view bounds based on the actual bytes reached by strides, improving accuracy for strided views. Views are required to start from contiguous elements along dimension 0, ensuring correct data access. Convolution windows in delta_net are created with all rows at once to prevent stride-related issues.
The llama.cpp release b12109 is now available, offering pre-compiled binaries for macOS, Linux, Windows, Android, and openEuler across various architectures and backends including CPU, CUDA, Vulkan, ROCm, OpenVINO, SYCL, and specialized hardware like Snapdragon. The release includes support for Apple Silicon and Intel macOS, multiple Linux distributions, and detailed setup guides for specific platforms.
The llama.cpp project has been updated to version 0.9.0, with new releases available for multiple platforms including macOS, Linux, Windows, Android, and iOS. The update includes support for various backends such as CUDA, Vulkan, ROCm, OpenVINO, SYCL, and Snapdragon-specific hardware. Some builds are marked as disabled or experimental, and detailed setup guides are provided for specific configurations.
arXiv:2610.37819v1 Announce Type: new Abstract: Quantum machine learning (QML) is increasingly explored for medical classification, yet its behaviour under subgroup-aware fairness criteria remains understudied. We present an empirical fairness audit of classical and quantum classifiers for medical imaging, assuming QML is not inherently fair. Two scenarios are evaluated: dermatological lesions on DDI (sensitive attribute: Fitzpatrick skin type) and chest X-rays on CheXpert-derived data (sensitive attribute: age group). Using ResNet-18 embeddings, we train SVM, MLP, QSVM and QCNN under a baseline and three mitigation settings: differentiable Equalized Odds (EO) regularisation, Gradient Reversal Learning (GRL), and both combined. Foundation models (PanDerm, EVA-X) serve as external reference
tests : adjust constants to reduce irrelevant warnings (#31254) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11954/llama-b11954-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11954/llama-b11954-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11954/llama-b11954-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11954/llama-b11954-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b1
cuda : add top-k MoE fusion for 384 experts (#31288) Assisted-by: Codex **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11953/llama-b11953-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11953/llama-b11953-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11953/llama-b11953-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11953/llama-b11953-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/dow
chat : stop qwen3-coder reasoning and content before template separators (#30557) * chat : strip trailing whitespace from parsed reasoning and content Assisted-by: Claude Fable 5.1 * chat : stop qwen3-coder reasoning and content before template separators Assisted-by: Claude * tests : remove the Qwen parser case from the chat session tests Assisted-by: Claude * common : handle partial prefix until match at end of generation --------- Co-authored-by: Zeshawn Zahid Co-authored-by: Alde Rojas **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11951/llama-b11951-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel
server : add realtime conversation and transcription on /v1/realtime (#31269) * server : add realtime conversation and transcription on /v1/realtime Co-authored-by: Xuan Son Nguyen Assisted-by: Claude Opus 5.5 * examples : default the realtime client to the server port 9931 Assisted-by: Claude Opus 5.5 * server : sample realtime responses right after their own decode Slots respond in turns on one context: sampling at the start of the next step read the logits of whichever slot decoded last, so concurrent sessions mixed their outputs. Assisted-by: Claude Opus 5.5 * move more stuff to server-realtime, making it self-contained * examples : fix the type check of the realtime client Assisted-by: Claude Opus 5.5 * examples : use the model card prompts in the realtime client Assisted-by: Claude O
model : add LiquidAI/d1-2B and LiquidAI/d1-omni-2B (#31275) * model : add LiquidAI/d1-2B and LiquidAI/d1-omni-2B Assisted-by: Claude Opus 5.5 * convert : keep the speech task templates out of the d1 GGUFs Assisted-by: Claude Opus 5.5 * small refactoring: no_batching --------- Co-authored-by: Xuan Son Nguyen **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11945/llama-b11945-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11945/llama-b11945-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b119
server: expose the reasoning effort levels of the chat template (#31271) * server: expose the reasoning effort levels of the chat template The capability probe renders a user turn for each candidate level (minimal, low, medium, high, xhigh, max) and for an unknown level. A level is honored when it renders without raising and differently from the unknown level, which shows the template fallback. Levels sharing a render are aliases and the highest one is kept. The caps serialize to JSON, and chat_template_caps in /props reports the list as reasoning_effort_levels, from lowest to highest. * jinja: keep the caps free of the json dependency The caps keep reporting their flags as a map, and the effort levels stay a plain member. common_chat_templates_get_caps() builds the JSON the server reports