Search headlines, summaries and articles to connect the dots.
The Metal kernel for ggml_tri has been updated to use a select operation instead of scaling by comparison, aligning its behavior with the CPU kernel. This change ensures that values outside the triangle are set to zero rather than becoming NaN when encountering ±inf. A new test, test_tri_special, validates this fix across all four triangle types, confirming it passes with the update but failed previously.
The release addresses the issue of invalid UTF-8 characters in arguments passed to PEG tool calls, improving input handling and robustness.
The ggml library has been updated to make tied selections in the radix Top-K algorithm deterministic on CUDA and Metal backends by replacing atomic compaction loops with a deterministic scheme using simdgroup scans and coalesced memory accesses. This change ensures byte-identical outputs across runs, improves performance by 20-45% on M5 Ultra hardware, and enables deterministic testing for Metal and ROCm. Redundant CUDA kernel resets were removed, and Vulkan/Metal changes were reduced or skipped.
A fix has been implemented to prevent crashes in the command-line interface when encountering invalid UTF-8 in prompts or files, with assistance from Claude. This update is part of the b12177 release of llama.cpp, which includes various binary distributions for macOS, Linux, Windows, Android, and other platforms.
The Vulkan implementation of the Fast Walsh-Hadamard Transform (FWHT) has been enhanced to support block widths up to 8192, using shared-memory shaders for larger blocks, and now includes F16 input capability via shader-float16, with pipelines created only when device limits allow. This update aligns Vulkan with CUDA's capabilities and improves efficiency by eliminating the need for F32-to-F16 conversions.
One underperforming GPU can slow an entire distributed training job. The node may pass basic health checks and show normal utilization, leaving operators to...
As artificial intelligence and accelerated computing continue to push the boundaries of what's possible, the latest NVIDIA CUDA release arrives to unlock...
The update introduces initial support for the LIGHTNING_INDEXER operation in the Hexagon backend, including optimizations for HMX prefill, q4_0 and f32 K support, pipelined HMX paths, improved DMA pipeline with canonical double buffering, and enhanced kernel parameter alignment. It also adds options to force HMX/HVX execution paths and supports dimension collapse.
vLLM supports Kimi K3.1 Pro on day 0: sparse attention on a KDA hybrid, a packed KV cache with FP8 or NVFP4 latents and MXFP4 index keys, token-sharded prefill, decode context parallelism, and fused low-latency decode.
tests : fix multi-seq split replay and partial-state cases for trained models (#31216) * tests : fix multi-seq split replay against trained models The rollback snapshot is captured by the graph that decoded the rolled-back tail, while the reference context decodes the same prefix with different batch shapes. The two states differ in the last bits and trained models amplify the difference with every replayed token. Compare only the first replayed token of each seq, then restore the rollback state into the reference context before the seq-1-only tail. Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * tests : require bit-exact seq-1 isolation in split replay The seq state save is not always bit-lossless - SWA-masked cells are not saved - so reload the rest
vLLM Semantic Router released d3-ultra, a 27B open-weight multimodal decision model. It reads text, images and video once, then returns probabilities for every answer, scoring 87.2 on the team’s SR-Text test. The post vLLM Semantic Router Releases d3-ultra: A 27B Open Multimodal Decision Model appeared first on MarkTechPost.
