How I Fight AI Brain Fry
Article URL: https://sergeykuhne.com/blog/fighting-ai-brain-fry Comments URL: https://news.ycombinator.com/item?id=50416363 Points: 83 # Comments: 56
AI NEWS JOURNAL
Search AI news
Search headlines, summaries and articles to connect the dots.
Show HN: FELF – AppImage-like single binaries without FUSE, root, or a runtime
Hello HN!I've always had a bad time redistributing programs that use shared libraries, AppImage in my opinion does too much, and a container is an overkill - not always integrates smoothly.I've moved to nix since long ago, but redistributing between nix and other distros or vice versa it's still a pain in the ass, more so if you want to run an alien app in nix or a nix app without the package manger in any other distro.For that, with the help of Claude, I've implemented a format that deduces at pack-time all the shared libraries a program needs, puts them, together with its own glibc and a patched dynamic linker into a single executable.This means that you don't actually need a runtime nor any special capability, it can run inside a FROM SCRATCH docker, alpine linux etc... And it also supports coreutils-like multiple programs accessible via symlinks, so it is a good fit for that purpose.You can just use:> felf pack $(which git) --output git.felfand you'll have already a git binary that works everywhere, no matter which distro you are building this from. This only packs the binary and their dependencies.Essentially, the effect is similar as if you LD_PRELOAD's all the libraries that it requires.In the end, we have been fighting these demons since long long ago, chroots/docker/appimage/flatpack/nix etc...So this xkcd applies: https://xkcd.com/927/I'm happy to share the project, I hope it inspires new discoveries/tools in this regard! Comments URL: https://news.ycombinator.com/item?id=50415219 Points: 2 # Comments: 0
Show HN:Free Web Tools
Article URL: https://nyneon.net/tools/ Comments URL: https://news.ycombinator.com/item?id=50418253 Points: 3 # Comments: 2
Show HN: Hyper Compress: The Best Open Source PDF Compression Engine
Article URL: https://github.com/alam00000/bentopdf-hyper-compress Comments URL: https://news.ycombinator.com/item?id=50416782 Points: 1 # Comments: 0
Show HN: LLM-cost-watch – Ask Claude why your LLM bill jumped
Article URL: https://github.com/rohith0707/llm-cost-watch Comments URL: https://news.ycombinator.com/item?id=50415785 Points: 1 # Comments: 0
Show HN: Record, branch, and compare command runs in stdlib-only Python
Hi, this is a package I developed for keeping track of short toy-model development cycles and explaining what changed between runs.The core idea: commands run through the package, and their declared parameters (or injected parameters with --substitute) and outcomes go into a dependency-free ledger stored in the project. Because that ledger lives in the project directory, it has worked well for passing context between different agents. I’ve also found some use for it in CI work. Comments URL: https://news.ycombinator.com/item?id=50417782 Points: 3 # Comments: 0
Show HN: Tab-league – archive Chrome tabs locally so closing them feels safe
Article URL: https://github.com/Shann5/tab-league Comments URL: https://news.ycombinator.com/item?id=50416248 Points: 1 # Comments: 0
ASRock Rack DMRD16-2L+ Spotted with 16 DDR5 Slots for Diamond Rapids
We check out the ASRock Rack DMRD16-2L+ which is a 16-channel Intel Diamond Rapids Xeon motherboard with 128x PCIe Gen6 lanes over MCIO The post ASRock Rack DMRD16-2L+ Spotted with 16 DDR5 Slots for Diamond Rapids appeared first on ServeTheHome.
b12191
The latest update to llama.cpp introduces enhanced MoE (Mixture of Experts) cache functionality, enabling computation of cache misses on the CPU with a configurable CPU share. A new tuning script, moe-cache-tune.py, automatically determines optimal cache size and CPU share settings based on system performance. The update also improves scheduling efficiency, removes redundant sync operations, and refactors code for better maintainability, while maintaining compatibility across multiple platforms including macOS, Linux, Windows, and Android.
b12192
The latest release of llama.cpp adds support for the MageVL-Cowork model and adjusts how rope frequencies are handled, now stored at the model level within clip_model instead of being copied into each clip_layer. The loader verifies that rope_freqs is a 32-bit floating-point array with dimensions matching d_head, ensuring compatibility with expected 4:6:6 splits in the computational graph.
b12193
The llama.cpp release b12193 includes a change to move log-mel computation to the cgraph, removes stray files, updates mtmd_audio_track to include source, and adds tests. It provides precompiled binaries for macOS, Linux, Windows, Android, and openEuler across various architectures and backends including CUDA, Vulkan, ROCm, OpenVINO, SYCL, and Snapdragon-specific platforms.
b12195
The llama.cpp release b12195 introduces a change to refuse device buffers that would exceed VRAM, improving memory management. It includes precompiled binaries for multiple platforms including macOS, Linux, Windows, Android, and openEuler, with support for various backends such as CUDA, ROCm, Vulkan, OpenVINO, SYCL, and others. Some builds are marked as disabled or experimental.
b12196
The release includes updates for SYCL backend with exact rounding support in CPY operations for Q4_1, Q5_1, and Q8_0 quantization formats.
b12197
The update introduces support for automatic loading of libhtp without relying on legacy environment variables, enhances path handling for non-ASCII directories and long paths, adds platform-wide support for FASTRPC_MANAGE_DSP_LIBRARY_PATH and GGML_HEXAGON_LIBHTP_DIR, maintains forward slashes in absolute paths on Android and Linux, ensures absolute path precedence with fallback mechanisms, and includes CDSP1,2,3 in the fallback chain for IQ10 compatibility.
b12198
The llama.cpp release b12198 includes a change to downgrade per-request proxy logs to trace level, along with numerous precompiled binaries for macOS, Linux, Windows, Android, and other platforms, supporting various backends like CUDA, Vulkan, ROCm, OpenVINO, and SYCL. Some builds are disabled or experimental, including openEuler and KleidiAI-enabled macOS versions.
b12199
The release addresses the issue of invalid UTF-8 characters in arguments passed to PEG tool calls, improving input handling and robustness.
b12200
Enhancements to OpenCL support in llama.cpp include adding new kernel functions for quantized matrix operations, improving small-n MoE GEMV performance across multiple quantization types, optimizing Adreno GPU kernel floor settings, renaming functions for clarity, and disabling bin FA for small verification batches.
b12201
The llama.cpp project has released version b12201, offering pre-compiled binaries for multiple platforms including macOS, Linux, Windows, Android, and iOS. The release includes support for various backends such as CUDA, Vulkan, ROCm, OpenVINO, SYCL, and Snapdragon hardware. Notably, the MoE cache feature is disabled for the draft model in this release. Additional resources include setup guides for Snapdragon devices and separate downloads for CUDA and OpenVINO libraries.
b12202
The llama.cpp v1.220.2 release introduces enhanced BF16 precision support across multiple components, aligning with reference implementations by moving specific operations like GLM4 family projections, MiMoVL vision matmuls, and Qwen3-TTS speaker encoder conv1d to BF16 where appropriate. Backends such as CUDA, Hexagon, OpenCL, WebGPU, and Vulkan now uniformly handle GGML_PREC_BF16 similarly to F32, while CPU, SpaceMIT, and ET backends extend flash attention support to BF16 with F32 accumulation. Flash attention tests now include BF16 cases, and the KQ multiplication in Llama is updated to request BF16 precision. A TODO remains in clip.cpp for future work.
b12207
A new fast path for d_state=64 has been added to the SSM_SCAN kernel in ggml-cuda, improving generation throughput for Zamba2-7B-Instruct from 11 to 15.1 tok/s—approximately a 35% increase—by eliminating CPU fallbacks for this state size. The change was verified with real-world Zamba2 hyperparameters and maintains correctness, though a separate 28% of scan calls still fall back to CPU due to an undiagnosed issue.