저장한 도구도구 추천
FOXAI SEARCH

전체 검색

뉴스 기사 218

coding-engineering

Workflow SDK now writes over WebSockets on Vercel

The Vercel Workflow SDK has updated to send event writes and stream writes over WebSocket by default starting with version 5.2.0, eliminating the need for one HTTP request per write. This change improves performance, reduces costs, and enables faster AI token streaming—demonstrated by a 4.2× speed improvement in benchmarks. No code changes are required beyond upgrading to workflow@5.2.0 or later. Existing streaming code remains compatible, and in-flight workflows continue on their original deployment. Users should consult the upgrade guide if migrating from version 4.x.

번역 서비스가 일시 중지되어 원문을 표시합니다.
Vercel
open-models

Your Experts Have Partners. There's HOPE for Keeping Them Together

HOPE, a second-order expert pruning method for Mixture-of-Experts (MoE) language models, accounts for pairwise expert interactions by optimizing a quadratic program based on an interaction matrix derived from calibration data. Unlike first-order methods that prune experts independently, HOPE preserves cooperative expert groups, leading to better performance—especially at high pruning rates like 50%. On benchmarks such as SWE-Bench Pro, HOPE outperforms state-of-the-art first-order methods like REAP by up to +6.1%, maintaining model quality while significantly reducing memory footprint. The method requires only a single calibration pass and efficient quadratic programming, with negligible overhead. It has been validated on Qwen3.5 and GLM-4.5 models, showing consistent improvements across diverse tasks and calibration sets.

번역 서비스가 일시 중지되어 원문을 표시합니다.
Hugging Face 官方博客
inference-systems

b12202

The llama.cpp v1.220.2 release introduces enhanced BF16 precision support across multiple components, aligning with reference implementations by moving specific operations like GLM4 family projections, MiMoVL vision matmuls, and Qwen3-TTS speaker encoder conv1d to BF16 where appropriate. Backends such as CUDA, Hexagon, OpenCL, WebGPU, and Vulkan now uniformly handle GGML_PREC_BF16 similarly to F32, while CPU, SpaceMIT, and ET backends extend flash attention support to BF16 with F32 accumulation. Flash attention tests now include BF16 cases, and the KQ multiplication in Llama is updated to request BF16 precision. A TODO remains in clip.cpp for future work.

번역 서비스가 일시 중지되어 원문을 표시합니다.
llama.cpp releases
inference-systems

b12208

The ggml library has been updated to make tied selections in the radix Top-K algorithm deterministic on CUDA and Metal backends by replacing atomic compaction loops with a deterministic scheme using simdgroup scans and coalesced memory accesses. This change ensures byte-identical outputs across runs, improves performance by 20-45% on M5 Ultra hardware, and enables deterministic testing for Metal and ROCm. Redundant CUDA kernel resets were removed, and Vulkan/Metal changes were reduced or skipped.

번역 서비스가 일시 중지되어 원문을 표시합니다.
llama.cpp releases
inference-systems

b12179

The update introduces support for 3D convolution (conv_3d) on Hexagon hardware by generalizing the existing conv_2d implementation. It leverages the HMX tiled path, folds kernel dimensions into a virtual row, and uses a shared solver with an is3d flag. The host computes unfold divisors to avoid device-side division, prevents DMA queue overflow, and improves tile size selection based on VTCM usage. Additional changes include handling arbitrary positive strides, preserving load/shuffle paths for small strides, and fixing compiler warnings.

번역 서비스가 일시 중지되어 원문을 표시합니다.
llama.cpp releases
inference-systems

b12180

The release adds the Kimi-K3.1-Pro text model to llama.cpp, replacing full-attention MLA layers in K3 with DSA (Dynamic Sparse Attention) layers featuring a lightning indexer. The indexer uses an additive mask over the cache and reuses sparse flash attention paths, while preserving existing KDA layers and MoE components. The model's architecture includes four indexer tensors per DSA layer, hybrid memory for caching, and maintains quantization integrity by excluding indexer parameters from quant overrides. Verified against HF checkpoint with identical PPL beyond context length, and supports multiple platforms including macOS, Linux, Windows, Android, and iOS.

번역 서비스가 일시 중지되어 원문을 표시합니다.
llama.cpp releases