전체 AI 소식

b11899

mtmd: mtmd_helper_get_capabilities 추가 (#31183) * mtmd: mtmd_helper_get_capabilities 추가 * cap audio_stream * 호환성을 위해 기존 API 복구 * audio_gen_spk_ref, text_gen **웹사이트:** - **인증서:** - **macOS/iOS:** - [macOS 애플 실리콘(arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11899/llama-b11899-bin-macos-arm64.tar.gz) - macOS 애플 실리콘(arm64, KleidiAI 활성화) [비활성화](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS 인텔(x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11899/llama-b11899-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11899/llama-b11899-xcframework.zip) **Linux:** - [Ubuntu x64(CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11899/llama-b11899-)

징둥 광고는 화웨이 쿤펑과 협력해 생성형 광고 추론 지연을 15% 단축했으며, 618 대규모 프로모션에서 핵심 시나리오 검증을 완료했습니다.

징둥 광고는 화웨이 쿤펑과 협력하여 링취우 인터넷을 통해 300TB 규모의 공유 메모리 풀을 구축하고 RPC 통신을 재구성함으로써, 엔드투엔드 추론 지연 시간을 15% 단축하고 KV 캐시와 RPC 성능을 크게 향상시켰으며, 2026년 618 대규모 프로모션의 핵심 시나리오에서 5% 트래픽을 통해 검증을 완료했습니다. 또한 이 협력은 서치 및 추천 등 검색·추천 시나리오에 긍정적인 파급 효과를 가져왔으며, SuperCache, Agent 샌드박스 등 애지네틱 AI의 최첨단 분야에도 적극적으로 진출하고 있습니다.

추천 이유：이 보도는 2026년 10월 26일에 tmtpost.com에서 발표된 독창적인 기술 뉴스를 바탕으로 하며, 실제 적용된 기술 협업을 중점적으로 다룹니다. 협력 주체(징둥 광고/화웨이 쿤펑), 기술 경로(링취우 상호 연결, 이중 캐시, UB 프로토콜 직접 연결 URMA), 정량적 성과(지연 시간 15% 감소, TP99 최적화 85% 이상), 검증 시나리오(618 대규모 프로모션 트래픽의 5%), 그리고 발전 계획(SuperCache, Agent 샌드박스) 등이 명확히 공개됩니다. 모든 핵심 사실은 자료 내에서 상호 확인 가능하며, 외부 정보의 추가나 주관적 추론은 포함되지 않습니다.

b11898

ggml-cpu : fix unused variables in arm arch (#31180) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11898/llama-b11898-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11898/llama-b11898-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11898/llama-b11898-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11898/llama-b11898-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11898/llama-

b11897

tests : refactor test-llama-archs to not link internal libllama symbols (#30880) test-llama-archs included src/llama-arch.h and src/llama-model-saver.h, which prevented building on Windows with shared libraries since the used symbols are not exported from the DLL. - declare llama_arch_* wrappers around the internal llm_ arch query functions in the llama-ext.h staging header, together with llama_arch_kv_name() and llama_arch_supports_save() - slim llama_model_saver down to the private implementation of llama_model_save_to_file(): the fixture-writer half used by the test moves into tests/test-llama-archs.cpp as a local kv_writer - replace the roundtrip saver usage with the public llama_model_save_to_file() writing to a temp path (also removes the tmpfile() fd leak and its Windows workaround)

b11896

ci : fix race in webgpu memset (#31162) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11896/llama-b11896-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11896/llama-b11896-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11896/llama-b11896-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11896/llama-b11896-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11896/llama-b11896-bin-ub

b11895

metal : stop overflow from happening in large matrix output offsets (#30072) * Metal : stop overflow from happening in large matrix output offsets Assisted-by: Codex * metal : widen batch offsets and MUL_MAT_ID token indices Assisted-by: Codex --------- Co-authored-by: feni6 <270037094+feni6@users.noreply.github.com> **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11895/llama-b11895-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11895/llama-b11895-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/dow

b11890

deepseekv4.1: follow-up review fixes (#31137) * kv-cache : use ggml_backend_tensor_copy to refill the DSV4 K mirrors * llama : move the Engram pad id to llama_hparams and the token map to llama_model The saver no longer needs the deepseek41 model class. * deepseek41 : drop the F32 src1 request on the HC mixes matmul * ggml : add softplus TODOs in the backends that still use log(1 + exp(x)) * convert : make the DSV4/DSV4.1 modify_tensors generators, drop the redundant vision filter * deepseek41 : clean up comments * deepseek41 : drop the F32 src1 requests on the router and indexer matmuls * convert : use format_tensor_name directly in DeepseekV4Model * s/comp_kv_src/comp_il_src/g * cont : fix TODOs for log1p --------- Co-authored-by: Georgi Gerganov **Website:** - **Attestations:** - **macO

칭화대학교와 홍콩중문대학교가 공동으로 오픈소스 AKV를 출시했습니다: LLM 에이전트를 위한 KV 캐시 엔지니어링 서비스 시스템

칭화대학교와 홍콩중문대학교의 연구진은 2026년 10월 26일, SGLang 기반으로 구축된 대규모 언어모델(LLM) 에이전트 전용 KV 캐시 엔지니어링 서버인 AKV를 오픈소스로 공개했습니다. 이 시스템은 기존의 텍스트 히스토리 요약 방식이 아닌, 에이전트가 스스로 KV 캐시를 적극적으로 버리고 재배치하도록 함으로써, 4개의 에이전트 벤치마크에서 추론 비용을 평균 27.7% 절감하고, 과제 평균 점수를 1.7점 상승시켰습니다. 해당 코드는 Apache-2.0 라이선스를 적용했으며, 현재 Qwen, GPT-OSS-120B, MiniMax-M2.7 등의 모델을 지원합니다.

추천 이유：이 성과는 LLM 에이전트의 효율적 추론을 위한 새로운 패러다임을 제시합니다. 텍스트 절단이나 요약에 의존하는 ‘컨텍스트 엔지니어링’과 달리, AKV는 KV 캐시 상태를 직접 조작하여 중복 프리필을 방지함으로써 장거리 상호작용에서의 KV 부하를 크게 완화합니다. 실제 실험 결과, 높은 동시성 환경에서 P95 지연 시간이 229.9초에서 30.6초로 감소하여 명확한 엔지니어링 가치와 재현 가능성을 입증했습니다. 해당 정보는 MarkTechPost가 발표한 권위 있는 기술 보고서에 근거하며, 게시일시(2026-10-26T07:56:45Z)가 현재 시각과 일치해 최신 공개된 진행 상황임을 나타냅니다.

b11889

cuda : use std::string instead of VLA for archName (#30463) * cuda : use std::string instead of VLA for archName Modernize the code (old C style) and fix the following warnings: ``` ggml/src/ggml-cuda/ggml-cuda.cu:180:19: warning: variable length arrays in C++ are a Clang extension [-Wvla-cxx-extension] 180 | char archName[archLen + 1]; | ^~~~~~~~~~~ ggml/src/ggml-cuda/ggml-cuda.cu:180:19: note: read of non-const variable 'archLen' is not allowed in a constant expression ggml/src/ggml-cuda/ggml-cuda.cu:179:9: note: declared here 179 | int archLen = strlen(devName); | ^ ``` Signed-off-by: Adrien Gallouët * Add a comment about archName Signed-off-by: Adrien Gallouët --------- Signed-off-by: Adrien Gallouët Signed-off-by: Adrien Gallouët **Website:** - **Attestations:** - **macOS/iOS:** - [ma

b11888

common, grammar: add recursive depth limit, make it a requirement (#31153) * common, grammar: add recursive depth limit, make it a requirement * catch more recursive functions * common : tidy up peg parser changes * grammar : reject candidates in a loop, without a depth limit The depth limit in llama_grammar_reject_candidates_for_stack counted the code points of a vocab token, so any token of 256 chars or more was always rejected under a grammar, such as the 512-space token of GLM-4.7-Flash or the long space tokens of Laguna-S-2.1. The rejection now walks the code points with its own stack of levels instead of recursing: there is no limit and no stack risk, whatever the length of a token from the GGUF, and the masks match master. * grammar : track rejected candidates as indices into the in

b11887

tests : adjust default threads for test-llama-context (#31157) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11887/llama-b11887-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11887/llama-b11887-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11887/llama-b11887-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11887/llama-b11887-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11

b11886

qwen: merge tensors on convert (#31028) * qwen: merge tensors on convert * convert: use generator functions * qwen4exp: remove conts and pre-scale **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11886/llama-b11886-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11886/llama-b11886-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11886/llama-b11886-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11886/llama-b11886-bin-ubuntu-x64.tar.gz

b11884

llama: fix exhausting CPU and MEM caused by dry sequence breaker (#31035) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11884/llama-b11884-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11884/llama-b11884-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11884/llama-b11884-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11884/llama-b11884-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/d

b11883

llama-quant : fix 4D tensor quantization using ggml_nrows() and name-based expert detection (#30820) * llama-quant : fix quantizing 4D tensors nrows_total was ne[1]*ne[2], so for a 4D tensor only the first ne[2] slices were quantized and the rest of the rows were dropped. Use ggml_nrows(tensor) to count all rows, including ne[3]. Fixes #29916 * llama-quant : detect expert tensors by name and fix imatrix size for 4D tensors The MXFP4_MOE path used ne[2] > 1 to detect MoE expert tensors. This is wrong for non-expert tensors with 3 or more dims, e.g. MLA projections (BailingMoE3) and 4D conv kernels. Use the tensor name instead: all tensors used by GGML_OP_MUL_MAT_ID are named *_exps or *_chexps. This also removes the BailingMoE3 special case. The imatrix holds one vector of ne[0] values per

b11882

grammar: limit recursion depth to prevent stack overflow (#29495) (#29755) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11882/llama-b11882-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11882/llama-b11882-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11882/llama-b11882-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11882/llama-b11882-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/

b11881

llama : respect no_alloc in all memory caches (#31087) * llama : respect no_alloc in memory caches Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) * llama : TODO: move no_alloc out of llama_hparams Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) * llama : TODO: move no_alloc out of llama_hparams Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11881/llama-b11881-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS

b11880

tools : reject truncated GGUF tensor data in merge (#31044) Co-authored-by: Shreya Agrawal **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11880/llama-b11880-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11880/llama-b11880-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11880/llama-b11880-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11880/llama-b11880-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/lla

b11879

CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/up rows (#29948) * cuda: fuse FFN gate/up and GLU in MMQ with interleaved gate/up rows For prompt processing, run the gate and up matmuls of a dense FFN as one mul_mat_q launch and apply the GLU in the write-back. - The loaders take the gate weights as a second pointer and fill the weight tile with up and gate rows interleaved in chunks of 8, so each thread holds an up value and the matching gate value (rows r and r + 8 of an MMA tile). - The MMA loop is unchanged: same tile, warp layout, vec_dot and one sum[] per thread. The fused write-back computes the GLU from registers and writes half the rows. - The fused path uses the regular tiling (no stream-k) with the column tiles innermost so that neighbouring blocks share weig

b11878

ggml-cuda: fuse qwen MRoPE + set-rows (#31025) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11878/llama-b11878-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11878/llama-b11878-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11878/llama-b11878-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11878/llama-b11878-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11878/llama-b11878

b11877

vulkan: Add a larger tile size for conv3d (coopmat2 only) (#29920) * vulkan: Add a larger tile size for conv3d (coopmat2 only) This adds a larger tile size, in practice it is only for coopmat2 where we don't need shared memory to store the output (would use more than 64KB). Add some additional representative backend tests. At least, this is what codex came up with from looking at sd.cpp Wan VAE. * only apply large tile size for newer driver versions **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11877/llama-b11877-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama