Skip to content
AI NEWS.ccAI正在发生 未来正在更新
Bookmarks
THE JOURNAL
AI NEWS JOURNAL
AI, IN PERSPECTIVE

Search AI news

Search headlines, summaries and articles to connect the dots.

Newest first

llama.cpp releases

b12208

The ggml library has been updated to make tied selections in the radix Top-K algorithm deterministic on CUDA and Metal backends by replacing atomic compaction loops with a deterministic scheme using simdgroup scans and coalesced memory accesses. This change ensures byte-identical outputs across runs, improves performance by 20-45% on M5 Ultra hardware, and enables deterministic testing for Metal and ROCm. Redundant CUDA kernel resets were removed, and Vulkan/Metal changes were reduced or skipped.

llama.cpp releases

b12176

The update qualifies calls to math functions like isnan and signbit with std:: to ensure compatibility with C++ standard library overloads, addressing issues on NetBSD 11.99.9/x86_64 with gcc 14.4.0. Additional C math function calls are standardized using std:: prefixes (e.g., std::fabs, std::copysign) instead of non-existent std::fabsf variants, improving accuracy by keeping computations in double precision. The change results in a minor behavioral difference with a relative error of at most 1.4e-7, well below test tolerances.

llama.cpp releases

b12188

The server update addresses slot processing order by ensuring parent slots are visited before child slots, preventing delays caused by child tasks landing on lower slot IDs. This change ensures all child tasks join the same batch after being woken by their parent. Additionally, new test tags have been introduced for monitoring: __TEST_TAG_WAIT_PARENT_SLOT__ for child slot waits, __TEST_TAG_PROMPT_CACHE_RESTORE__ for prompt cache restores, and __TEST_TAG_PROMPT_CACHE_FULL_RESTORE__ when a cached prompt is fully contained in a new one based on token counts.

llama.cpp releases

b12170

The llama.cpp v1.21.70 release includes enhancements to the moe-cache system, introducing an LRFU/MQ-inspired hybrid cache with improved token handling, key computation, and decay calculations. Changes include adjusting the half-life parameter, increasing the number of classes to 8, refining score computation based on age, removing redundant branches, and improving documentation around unused slots and sorting. The release also features updated build binaries for multiple platforms including macOS, Linux, Windows, Android, and openEuler, with support for CUDA, Vulkan, ROCm, OpenVINO, SYCL, and other backends.

vLLM · 官方技术博客

Migrating vLLM to the LibTorch Stable ABI

How vLLM migrated its CUDA build and compiled dependencies to the LibTorch Stable ABI, enabling compatible PyTorch upgrades without rebuilding vLLM.

arXiv cs.CL

Where Should a KV-Cache Repair Budget Go? Distractor Structure and Contextual Compatibility in Non-Prefix Reuse

arXiv:2611.06397v1 Announce Type: new Abstract: Non-prefix KV-cache reuse encodes retrieved chunks independently and concatenates their caches. To find where a small repair budget should be spent, we vary the chunks around the answer-bearing one (the target) across three 7B-8B models and four datasets of dialogue memory, open-domain QA and API documentation. Reuse loses 19-100% of the context-dependent questions that full prefill answers, and it loses more when the other chunks are of the same kind as the target. Because the recoverable damage is spread over these other chunks, restoring a few of them recovers little, and the budget is better spent on the target. This works through contextual compatibility: recomputing the target against the reused cache beats inserting its full-prefill KV in all seven settings tested, although the recomputed KV stays far from the full-prefill KV. Question attention over reused caches, however, selects the target in at most 18.8% of development questions. A retriever finds it more often, and recomputing its top chunk after moving it next to the question significantly beats InfoFlow-style reordering at the same 5% budget in six of nine open-domain QA and API-documentation settings. On dialogue memory, though, recomputing chunks never significantly beats the token-level selector chosen for each setting.

llama.cpp releases

b12128

The fix addresses intermittent CI failures in TOPK_MOE tests caused by near ties in logits, particularly with uniform random inputs and narrow-range gating functions like sqrt_softplus. The solution involves redrawing logits until group selection and top-k decisions have a relative margin of 1e-4 in double precision, and constructing logits by inverting the gating function using a grid-based approach to ensure consistent expert selection across backends.

arXiv cs.CV

From Tokens to Trajectories: Parameter-Efficient Continuous Adaptation of Unified Multimodal Models

arXiv:2611.05451v1 Announce Type: new Abstract: While discrete tokenization in Unified Multimodal Models (UMMs) establishes strong semantic reasoning, it introduces a strict quantization bottleneck that degrades fine-grained visual details, often resulting in structurally accurate but artifact-heavy images. We introduce FlowShift, a parameter-efficient post-training framework that bypasses this bottleneck by converting pre-trained discrete UMMs into continuous rectified-flow generators. Instead of discarding the backbone's semantic priors, FlowShift replaces discrete visual interfaces with continuous latent projections and applies LoRA to the attention layers. To efficiently boost visual quality without semantic drift, we propose Dual-Objective Velocity Vector Distillation (DOVD), which aligns the model with a high-fidelity continuous teacher and ground-truth continuous trajectories. This approach drastically improves perceptual quality while preserving prompt adherence and reasoning. When applied to leading discrete UMMs, FlowShift significantly improves COCO-30K FID and aesthetic scores, while retaining over 99.5% of multimodal understanding capabilities. Ablations confirm DOVD drives generation quality. Ultimately, FlowShift achieves high-fidelity continuous generation without sacrificing pre-trained semantic grounding.

llama.cpp releases

b11919

opencl: refine mul mat id -1 index (#31170) Assisted-by: Codex **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11919/llama-b11919-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11919/llama-b11919-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11919/llama-b11919-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11919/llama-b11919-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11

Timeline and related coverage

Ollama

Reduce hallucinations with Bespoke-Minicheck

Bespoke-Minicheck is a new grounded factuality checking model developed by Bespoke Labs that is now available in Ollama. It can fact-check responses generated by other models to detect and reduce hallucinations.

Ollama

Llama 3 is not very censored

Compared to Llama 2, Llama 3 feels much less censored. Meta has substantially lowered false refusal rates. Llama 3 will refuse less than 1/3 of the prompts previously refused by Llama 2.