Search headlines, summaries and articles to connect the dots.
A recent update to llama.cpp addresses a critical issue by rejecting non-object session updates in the server, preventing malformed realtime session data from causing uncaught exceptions and server termination. The release includes binaries for multiple platforms including macOS, Linux, Windows, Android, and iOS, with support for various backends like CUDA, Vulkan, ROCm, OpenVINO, and SYCL.
The update introduces optimized Metal kernel usage for matrix multiplication with K values between 32 and 64, and adds a new kernel for BF16 x BF16 matrix multiplication, improving performance on Apple Silicon devices.
The latest update to llama.cpp (v1.2140) adds support for the `truncate=false` parameter in YaRN, enabling more accurate frequency interpolation for models like gpt-oss and certain PLaMo-3 variants by preserving non-integer interval boundaries. The change includes updates to GGML operations and CPU backend handling, with CUDA backend support, new tests, and refactoring of RoPE helpers into internal headers. Default behavior remains `truncate=true` for backward compatibility.
The latest update to llama.cpp introduces reference-counted checkpoints instead of deep-copies, reducing memory overhead and improving efficiency. This change adapts the slot save/restore checkpoint mechanism to shared_ptr checkpoint lists and common_state_data buffers, with contributions from multiple developers and AI assistants.
A fix was implemented to verify the combined K+V f16 scratch size in the Flash Attention dequantization path for Vulkan, addressing issue #30048.
The Vulkan implementation of the Fast Walsh-Hadamard Transform (FWHT) has been enhanced to support block widths up to 8192, using shared-memory shaders for larger blocks, and now includes F16 input capability via shader-float16, with pipelines created only when device limits allow. This update aligns Vulkan with CUDA's capabilities and improves efficiency by eliminating the need for F32-to-F16 conversions.
The update introduces CUDA optimizations including linearizing the mixture-of-experts (MoE) data layout for MMQ/MMF, applying tile size optimizations more broadly, and addressing a half-precision (F16) performance regression.
The llama.cpp v1.21.70 release includes enhancements to the moe-cache system, introducing an LRFU/MQ-inspired hybrid cache with improved token handling, key computation, and decay calculations. Changes include adjusting the half-life parameter, increasing the number of classes to 8, refining score computation based on age, removing redundant branches, and improving documentation around unused slots and sorting. The release also features updated build binaries for multiple platforms including macOS, Linux, Windows, Android, and openEuler, with support for CUDA, Vulkan, ROCm, OpenVINO, SYCL, and other backends.
The update introduces batched tensor functions `get_tensor_n` and `set_tensor_n` for WebGPU, along with fixes for the editor-config checker and removal of an outdated comment.
The llama.cpp release b12174 introduces the ability to generate RC files directly without requiring a win-app.rc.in file, alongside a comprehensive set of precompiled binaries for macOS, Linux, Windows, Android, and openEuler across various architectures and backends including CUDA, Vulkan, ROCm, OpenVINO, SYCL, and Snapdragon-specific hardware. The release also includes a standalone UI package.
arXiv:2611.07069v1 Announce Type: new Abstract: Reinforcement learning (RL) in Partially Observable Markov Decision Processes (POMDPs) is computationally challenging due to the curse of long-range dependence on history. Fortunately, recent studies have shown that a range of POMDPs that satisfy the filter stability assumption can be well approximated by a superstate Markov Decision Process (MDP) based on a truncated history in a finite-memory window, with a value gap that decays with the window length. However, existing finite-time convergence analyses on such superstate MDP surrogates are either limited to over-simplified linear function approximation or rely on an i.i.d. sampling oracle that is violated by POMDP trajectories, both of which severely limit the applicability of these theoretical results in practice. This motivates us to bridge the gap by investigating RL in POMDPs with nonlinear neural value function approximation. Our main contributions are threefold: i) We first establish a new finite-state MDP point-wise approximation error bound, which lays the foundation for our RL algorithm design and analysis in POMDPs; ii) By applying Temporal-Difference (TD) learning to POMDP-generated superstate transitions and targeting the Bellman fixed point of the superstate MDP, we prove a finite-time critic error bound with nonlinear neural value function approximation; and iii) By combining the neural critic function approximation error bound with the policy mirror descent framework, we obtain a finite-time sublinear regret against the optimal POMDP value. To the best of our knowledge, such a finite-time sublinear regret based on neural function approximation is the first in the literature for finite-memory POMDPs. Our numerical experiments also verify our theoretical results.
OpenAI introduces BioBasicsBench, a 100-task benchmark measuring how well AI agents interpret raw experimental data and handle everyday wet-lab science.
I built it to replace the repetitive workflows of moving tabs and copying indicators across many tools and tabs for every investigation. You use your own API keys, and functions like decoding, identifying commands, and man pages all run locally with no LLM.Please let me know what you think, you can try it here right now. More information about it can be found here: ceplang.org/start.htmlAs well as a short demo here: https://youtu.be/bzGGigpeiXg Comments URL: https://news.ycombinator.com/item?id=50391790 Points: 1 # Comments: 0
Article URL: https://github.com/smokeeaasd/flawless-ui-ux Comments URL: https://news.ycombinator.com/item?id=50381846 Points: 1 # Comments: 0
English isn’t my first language, sorry for grammar mistakes.I usually use Windows and Android, and these two platforms can be linked via Microsoft's "Link to Windows", or "Phone Link" program. So, you can easily use some features such as notifications, messages, screen mirroring, and clipboard synchronization on Windows from your Android phone.Recently, I changed my personal computer to macOS from Windows, and it made me need a similar program. I developed this program to run on macOS while still using the "Link to Windows" app on Android phone.It supports most features of the Windows client (notifications, messages, photos, mirroring, file transfer, etc.), except calling.The license is MIT. This project is an unofficial client unrelated to Microsoft and has no affiliation with them. Comments URL: https://news.ycombinator.com/item?id=50389540 Points: 1 # Comments: 0
Chris Alexiuk has a simple rule about open source AI: If you believe in it, be loud about it. “We should be shouting about this technology from the rooftops,” said Alexiuk, a Toronto-based senior product research engineer in deep learning at NVIDIA who specializes in making open large language models (LLMs) practical for working developers. […]
Fixed get_max_size to return maxBufferSize and improved the memset shader to handle buffer sizes exceeding the maximum binding size, preventing issues with whole-buffer bindings in memset and clear operations.
The llama.cpp release b12116 introduces support for the drama-3 text-to-speech model, includes cleanup improvements, and ensures the drama3 helper stops at the codebook tower limit. This update provides binaries for multiple platforms including macOS, Linux, Windows, Android, and iOS, with support for various backends such as CUDA, Vulkan, ROCm, OpenVINO, and SYCL. Some builds remain disabled or require additional setup.
The update enhances OpenCL performance for decode operations by optimizing workgroup usage for q4_K gate up and GLU, and adds support for Flash Attention (FA) in various configurations including GQA 2, 6, and 16 with different head sizes, improving efficiency on specific hardware setups.
A fix was implemented in llama.cpp to correct the calculation of on-device sequence state copy sizes for quantized key-value (KV) caches, ensuring accurate copying of data blocks and preventing crashes or incorrect state restoration. The update adjusts the element count computation using proper block size handling and removes redundant code elements.
