Skip to content
AI NEWS.ccAI正在发生 未来正在更新
Bookmarks
THE JOURNAL
AI NEWS JOURNAL
AI, IN PERSPECTIVE

Search AI news

Search headlines, summaries and articles to connect the dots.

Newest first

llama.cpp releases

llama.cpp · b12048

sycl : fast ggml_sycl_comm_exchange() on Intel Arc Pro B70s (#30956) * Update ggml-sycl.cpp Increases -sm tensor prompt eval t/s by +19% with zero PPL change. * Update ggml-sycl.cpp Replace first pair of wait() calls with ext_oneapi_submit_barrier(). The wait() calls was discovered to cause MTP degradation. * Update ggml-sycl.cpp Adding comm_exchange function pointer to switch between 2 and 4 buffer ggml_sycl_comm_exchange versions @ initialization time (versus hot path) controlled by SYCL_ENABLE_DEFAULT_CONTEXTS=1 setting. * Update ggml-sycl.cpp adding back 2nd half of 4-buffer free. * Update ggml-sycl.cpp Removing extra whitespace. * Update ggml-sycl.cpp Excluding Windows until it is validated on Windows. **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](h

Timeline and related coverage

MIT Technology Review · AI

Stop counting tokens, start counting what ships

When a developer gives an AI agent a coding task, the agent does more than answer once. It plans, generates code, runs tests, and repeats the process to correct errors. A single task can use 10 to 50 times more tokens than a basic chat prompt. The more useful the agent becomes, the more calls…

Media
Timeline and related coverage

llama.cpp releases

b11980

[SYCL] add DPAS flash attention kernel for quantized KV cache (#31125) * sycl : add DPAS flash attention kernel for quantized KV cache Decode and small batches (1 to 31 query tokens) now run flash attention straight on q4_0 and q8_0 KV blocks, instead of scalar dequant or converting the whole cache to f16 per call. Q.K uses s8 x s4 and s8 x s8 DPAS on XMX, V.P stays f16. dpas.s4.s8 is inline vISA since joint_matrix has no s4. Covers head sizes 64 to 512, GQA up to 32 and attention sinks. Prompts stay on the existing kernels. GGML_SYCL_FA_DPAS=0 turns it off. B580, Llama 3.1 8B Q4_K_M, q4_0 KV, tg128: 31.3 to 54.2 t/s at 16K context. FLASH_ATTN_EXT 4582/4584, same 2 failures as master. Assisted-by: Claude * sycl : 192 head dropped, Wildcat Lake added to docs Assisted-by: Claude * sycl : str

Timeline and related coverage

llama.cpp releases

b11946

server : add realtime conversation and transcription on /v1/realtime (#31269) * server : add realtime conversation and transcription on /v1/realtime Co-authored-by: Xuan Son Nguyen Assisted-by: Claude Opus 5.5 * examples : default the realtime client to the server port 9931 Assisted-by: Claude Opus 5.5 * server : sample realtime responses right after their own decode Slots respond in turns on one context: sampling at the start of the next step read the logits of whichever slot decoded last, so concurrent sessions mixed their outputs. Assisted-by: Claude Opus 5.5 * move more stuff to server-realtime, making it self-contained * examples : fix the type check of the realtime client Assisted-by: Claude Opus 5.5 * examples : use the model card prompts in the realtime client Assisted-by: Claude O

Timeline and related coverage

llama.cpp releases

b11928

model: Add MiniCPM-OCR-Jev (#30980) * model : add MiniCPM-OCR-Jev support The model is a Qwen3.5-VL trunk with a value head read at the last token: it scores an OCR text against an image rendered from it, it does not generate. Add the openbmb-minicpm-ocr-jev decision type, served by POST /v1/systemone. The class scores are the pooled output of the sequence, read with last pooling, so the conversion turns score.weight into cls.output and sets embedding_length_out. The checkpoint has no LM head, so text generation is rejected instead of using the output weights duplicated from the token embeddings. The task is fixed, so its prompt is entirely in the systemone template, the same way h2o-lightning keeps its default there. The processor of the checkpoint allows up to 16384 merged tokens per ima

Timeline and related coverage

llama.cpp releases

b11879

CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/up rows (#29948) * cuda: fuse FFN gate/up and GLU in MMQ with interleaved gate/up rows For prompt processing, run the gate and up matmuls of a dense FFN as one mul_mat_q launch and apply the GLU in the write-back. - The loaders take the gate weights as a second pointer and fill the weight tile with up and gate rows interleaved in chunks of 8, so each thread holds an up value and the matching gate value (rows r and r + 8 of an MMA tile). - The MMA loop is unchanged: same tile, warp layout, vec_dot and one sum[] per thread. The fused write-back computes the GLU from registers and writes half the rows. - The fused path uses the regular tiling (no stream-k) with the column tiles innermost so that neighbouring blocks share weig

Timeline and related coverage