Search headlines, summaries and articles to connect the dots.
Seekable OCI (SOCI) indexing for Amazon SageMaker AI training and processing jobs takes full container image download off the critical path. SageMaker AI lazy-loads your image to start containers in seconds instead of minutes, with no changes to your job definition. See the benchmarks and how to adopt it.
Mirendil released Fern 0.1, its first model, pretrained from scratch with a 10M-token context window. It targets agentic AI R&D work, scores 100% on RULER single needle at 10M, but trails frontier models on agentic coding. The post Mirendil Launches Fern 0.1, Its First Model, With 10M-Token Context appeared first on MarkTechPost.
## What's Changed * model_prefetch: syncrhonize all graph cleanups by @rattus128 in https://github.com/Comfy-Org/ComfyUI/pull/17178 * Add rope scaling support to MOSS-image by @kijai in https://github.com/Comfy-Org/ComfyUI/pull/17172 * [Partner Nodes] Renew partner-node sign-in tokens in API nodes by @dante01yoon in https://github.com/Comfy-Org/ComfyUI/pull/16943 * [Partner Nodes] feat(HeyGen): add HeyGen Voice instant clone and text-to-speech by @bigcat88 in https://github.com/Comfy-Org/ComfyUI/pull/17043 * [Partner Nodes] feat(ClarityAI): add Crystal Video Upscaler node by @bigcat88 in https://github.com/Comfy-Org/ComfyUI/pull/17192 * [Partner Nodes] feat(BFL): add 2K/4K resolutions to FLUX 3 Video nodes by @bigcat88 in https://github.com/Comfy-Org/ComfyUI/pull/17200 * Support MOSS-image
When you’re speaking to a device, you expect text to appear as you talk. That makes streaming automatic speech recognition (ASR) a different challenge from batch transcription, where the model... The post Optimizing Streaming Speech Recognition on Arm® Edge Devices with ExecuTorch appeared first on PyTorch.
Fleet-wide computer vision needs a model per camera, not one model for the whole fleet. This post presents a working Amazon SageMaker SDK v3 reference architecture that trains, quality-gates, compiles per-camera models with SageMaker Neo, and versions thousands of them, with an AWS Step Functions layer pacing fleet-scale retraining within account quotas.
What is “night” to a machine?
Researchers from Stanford University and the University of California, Berkeley have introduced TokenGate—a plug-and-play, learnable token-pruning module designed for existing large language models (LLMs). This approach accelerates the first-token generation time (TTFT) for long-context prefill by 1.4 to 11.3 times without sacrificing accuracy, while reducing peak memory usage by 1.5 to 3.4 times. The associated code and checkpoints have been made open source.
On October 24, 2026, Sebastian Raschka published an eight-part, over-11-hour hands-on video course on Python and PyTorch in Ahead of AI magazine (magazine.sebastianraschka.com), systematically covering the evaluation of inference models, scaling during inference, reinforcement learning (RLVR), and knowledge distillation, with all methods implemented using the pre-trained Qwen3-0.6B model.
Why it matters：This course is the author’s advanced technical educational offering, following “Building Large Language Models from Scratch.” It focuses on the core evolutionary direction of today’s mainstream LLMs—enhancing inference capabilities—covering the entire technical pipeline from setting up the foundational environment to GRPO reinforcement learning and self‑distillation. The author explicitly states that all implementations are “hand‑crafted from scratch,” fully reproducible, and highly instructive with significant engineering reference value.
AWS has released a technical solution that leverages the Kokoro-82M text-to-speech model hosted on SageMaker, using bidirectional WebSocket to enable AI agents to generate text and synthesize speech simultaneously, significantly reducing end-to-end speech response latency. The solution offers two audio back‑transmission methods—Base64‑SSE and unified binary transmission—and is explicitly labeled as “proof-of-value,” meaning it has not yet reached production‑ready status.
This article is the first in a series of technical blog posts jointly published by Hugging Face and Amazon, systematically introducing how to use TorchNeuron on AWS Trainium AI accelerator chips for PyTorch model inference performance analysis. It covers the principles of Neuron backend integration, methods for configuring torch.profiler, the use of NeuronConfig and NeuronProfiler tools, and visualized tracing with Neuron Explorer. Additionally, using RMSNorm as an example, it compares the device‑time differences among three implementations: plain PyTorch, the built-in torch.rms_norm fusion, and torch.compile’s automatic fusion.
i14 has launched the Laplace 3B model with 3.028 billion parameters, which replaces Softmax attention with Gated Linear Attention (GLA). During inference, its per-sequence recursive state remains constant at 29.5 MB, independent of context length. The model was trained and validated on a single AWS trn2.48xlarge instance (equipped with 16 Trainium 2 chips), achieving a training throughput of 120,000 tokens per second and an inference throughput of 11,544 output tokens per second (with 1,280 concurrent sequences). All performance metrics are based on internal measurements conducted by i14 and the AWS GenAIIC team.
Hugging Face has released TorchTPU, providing native TPU backend support for PyTorch, enabling libraries such as Transformers and Diffusers to run directly on TPUs without relying on torch_xla or migrating to JAX.
PyTorch 2.15 brings a major MPS backend upgrade to Apple Silicon Macs, natively supporting FP8 tensor operations on macOS for the first time, boosting MoE model decoding speed by 5–6.5 times, reducing peak video VAE memory usage from 270 GiB to 8 GiB, and migrating numerous operators from MPSGraph to dedicated Metal kernels.
Hugging Face, in collaboration with AWS, has launched TorchNeuron—a PyTorch-native Trainium backend—available for public beta starting October 20, 2026. This technology enables mainstream libraries such as Transformers and Diffusers to run directly on Trainium chips without any modifications, significantly enhancing the usability and compatibility of text-generation and image-generation models.
Profiling in PyTorch (Part 2): From nn.Linear to a Fused MLP
