Skip to content
AI NEWS JOURNAL
AI, IN PERSPECTIVE

Search AI news

Search headlines, summaries and articles to connect the dots.

Newest first

b12191

The latest update to llama.cpp introduces enhanced MoE (Mixture of Experts) cache functionality, enabling computation of cache misses on the CPU with a configurable CPU share. A new tuning script, moe-cache-tune.py, automatically determines optimal cache size and CPU share settings based on system performance. The update also improves scheduling efficiency, removes redundant sync operations, and refactors code for better maintainability, while maintaining compatibility across multiple platforms including macOS, Linux, Windows, and Android.

b12192

The latest release of llama.cpp adds support for the MageVL-Cowork model and adjusts how rope frequencies are handled, now stored at the model level within clip_model instead of being copied into each clip_layer. The loader verifies that rope_freqs is a 32-bit floating-point array with dimensions matching d_head, ensuring compatibility with expected 4:6:6 splits in the computational graph.

b12193

The llama.cpp release b12193 includes a change to move log-mel computation to the cgraph, removes stray files, updates mtmd_audio_track to include source, and adds tests. It provides precompiled binaries for macOS, Linux, Windows, Android, and openEuler across various architectures and backends including CUDA, Vulkan, ROCm, OpenVINO, SYCL, and Snapdragon-specific platforms.

b12195

The llama.cpp release b12195 introduces a change to refuse device buffers that would exceed VRAM, improving memory management. It includes precompiled binaries for multiple platforms including macOS, Linux, Windows, Android, and openEuler, with support for various backends such as CUDA, ROCm, Vulkan, OpenVINO, SYCL, and others. Some builds are marked as disabled or experimental.

b12196

The release includes updates for SYCL backend with exact rounding support in CPY operations for Q4_1, Q5_1, and Q8_0 quantization formats.

b12197

The update introduces support for automatic loading of libhtp without relying on legacy environment variables, enhances path handling for non-ASCII directories and long paths, adds platform-wide support for FASTRPC_MANAGE_DSP_LIBRARY_PATH and GGML_HEXAGON_LIBHTP_DIR, maintains forward slashes in absolute paths on Android and Linux, ensures absolute path precedence with fallback mechanisms, and includes CDSP1,2,3 in the fallback chain for IQ10 compatibility.

b12198

The llama.cpp release b12198 includes a change to downgrade per-request proxy logs to trace level, along with numerous precompiled binaries for macOS, Linux, Windows, Android, and other platforms, supporting various backends like CUDA, Vulkan, ROCm, OpenVINO, and SYCL. Some builds are disabled or experimental, including openEuler and KleidiAI-enabled macOS versions.

b12199

The release addresses the issue of invalid UTF-8 characters in arguments passed to PEG tool calls, improving input handling and robustness.

b12200

Enhancements to OpenCL support in llama.cpp include adding new kernel functions for quantized matrix operations, improving small-n MoE GEMV performance across multiple quantization types, optimizing Adreno GPU kernel floor settings, renaming functions for clarity, and disabling bin FA for small verification batches.

b12201

The llama.cpp project has released version b12201, offering pre-compiled binaries for multiple platforms including macOS, Linux, Windows, Android, and iOS. The release includes support for various backends such as CUDA, Vulkan, ROCm, OpenVINO, SYCL, and Snapdragon hardware. Notably, the MoE cache feature is disabled for the draft model in this release. Additional resources include setup guides for Snapdragon devices and separate downloads for CUDA and OpenVINO libraries.

b12202

The llama.cpp v1.220.2 release introduces enhanced BF16 precision support across multiple components, aligning with reference implementations by moving specific operations like GLM4 family projections, MiMoVL vision matmuls, and Qwen3-TTS speaker encoder conv1d to BF16 where appropriate. Backends such as CUDA, Hexagon, OpenCL, WebGPU, and Vulkan now uniformly handle GGML_PREC_BF16 similarly to F32, while CPU, SpaceMIT, and ET backends extend flash attention support to BF16 with F32 accumulation. Flash attention tests now include BF16 cases, and the KQ multiplication in Llama is updated to request BF16 precision. A TODO remains in clip.cpp for future work.

b12207

A new fast path for d_state=64 has been added to the SSM_SCAN kernel in ggml-cuda, improving generation throughput for Zamba2-7B-Instruct from 11 to 15.1 tok/s—approximately a 35% increase—by eliminating CPU fallbacks for this state size. The change was verified with real-world Zamba2 hyperparameters and maintains correctness, though a separate 28% of scan calls still fall back to CPU due to an undiagnosed issue.

b12208

The ggml library has been updated to make tied selections in the radix Top-K algorithm deterministic on CUDA and Metal backends by replacing atomic compaction loops with a deterministic scheme using simdgroup scans and coalesced memory accesses. This change ensures byte-identical outputs across runs, improves performance by 20-45% on M5 Ultra hardware, and enables deterministic testing for Metal and ROCm. Redundant CUDA kernel resets were removed, and Vulkan/Metal changes were reduced or skipped.

Show HN: An open-source IDE for software architecture

Now, with this open-source tool everyone can learn,build,share,and validate the system design.This is next step, drawing design on blank page to making it a workflow .Do explore and let me know how it is... Comments URL: https://news.ycombinator.com/item?id=50412135 Points: 1 # Comments: 0

Show HN: Coldframe, a honeypot that poses as a supermarket fridge controller

I built Coldframe, a honeypot that pretends to be a supermarket refrigeration controller. Inspired by the events on industrial fridge installations in the past weeks, I wanted to see what people would try against a convincing fake.To learn more about this I built a honeypot that resembled perfectly the Danfoss AK-SM 800A and Dixell XWEB interfaces. Both of them are recognized on Shodan as real ones (an not as honeypots). It’s one Python file using only the standard library, and it builds on eppser/danfoss-ak-sm-decoy and public research by Synacktiv, Claroty and Ironhackers.Things that were fun to get right:- Matching real units. I used Shodan records of real devices to match header order, page titles, certificate fields and TLS versions. The Danfoss persona sends nginx headers in the same order as a real AK-SM 800A. The Dixell persona sends no Server header and pads its page to exactly 8,202 bytes.- Recognising exploit attempts. It recognises CVE-2025-41450, the Danfoss “code of the day” login bypass, and records whether the submitted code is correct for today. It also tags attempts targeting three other Danfoss CVEs and the XWEB vulnerabilities.- Letting them in. Default logins open a fake store dashboard with defrost buttons, so you see the second move, not just the first. Nothing behind it is real.Two instances have been live for a day. So far they’ve only seen generic scanners and Shodan’s crawler, which is called refrigerator.census.shodan.io.Feedback welcome, especially from anyone who has seen these devices in the field! Comments URL: https://news.ycombinator.com/item?id=50406292 Points: 15 # Comments: 1

Show HN: Nino – Fastest C# serializer, safe on untrusted input, also TS/Go/Rust

Nino is a source-generated binary serializer for C#, and as far as we know the fastest one. In the same BenchmarkDotNet run on GitHub runners against MemoryPack 1.21.4, it serializes classes and polymorphic types 3.5–7.2x faster and deserializes them 1.7–2.1x faster (arrays of unmanaged structs are on par). Deserializing into existing objects allocates nothing. Reproduce: https://nino.xgamedev.net/en/docs/v4/benchmarksIt is built to read bytes you do not trust: lengths are checked before allocating, depth is bounded, hash flooding is limited, and the read paths are fuzzed in CI.The bytes are a spec, not a memory dump, so CoreCLR, Mono, IL2CPP and 32-bit hosts write the same bytes, and runtimes for TypeScript, Go and Rust pass the same conformance corpus.Trade-offs: positional by default (VersionTolerant or Keyed for stored data), a 4-byte type id on polymorphic values, no System.Type, needs C# 9 and Roslyn 4.3+.Docs: https://nino.xgamedev.net Comments URL: https://news.ycombinator.com/item?id=50413601 Points: 1 # Comments: 0