$ articles --topic=inference
Inference
75 AI articles about Inference.
Speed Up Image Generation with FLUX 3 Image Klein on NVIDIA Blackwell GPUsLearn how to reduce diffusion model inference latency when running FLUX Image Klein on NVIDIA Blackwell GPUs. The piece covers why sequential passes and high-resolution attention slow generation in design and image editing tools, and how to speed up the pipeline.Model Releases · via NVIDIA Technical Blog · 1d ago1 minAccelerating Reinforcement Learning Weight Refit with the Reshard CollectiveLearn how the Reshard collective speeds up weight refit in reinforcement learning post-training, where updated policy weights must be synced from training GPUs to separate inference GPUs in a disaggregated setup. The article covers why this transfer becomes a bottleneck and how the new approach reduces it.Inference · via NVIDIA Technical Blog · 1d ago1 minSageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpointsExplains how Amazon SageMaker AI now uses NVIDIA Multi-Instance GPU to split one physical GPU into isolated partitions, letting you host more models per endpoint and choose right-sized GPU capacity. It covers the API fields AcceleratorPartitionLayouts and AcceleratorPartitionType, plus the trade-offs and measured results.Open Models · via AWS Machine Learning · 1d ago8 minAgents Are Going AsyncExplains how async subagents let an agent stay responsive while background workers handle long-running tasks. Covers how the orchestrator, notifications, and approvals work in Deep Agents.Agents · via LangChain · 1d ago1 minAI SDK v6 Crash CourseThis crash course walks through the AI SDK v6, showing how to build AI-powered applications from the ground up. You will learn the core concepts and the practical steps needed to start developing with it quickly.Agents · via AI Hero (Matt Pocock) · 1d ago1 minZero Mismatch, Simple Async RL: REINFORCE Zero, DeepSeek-V4.1 and Kimi K3.1Explains how asynchronous reinforcement learning can train on stale model weights, and how aligning rollout and training kernels bit for bit, together with unclipped importance sampling, corrects that lag. Covers zero-mismatch RL applied to DeepSeek-V4.1-Flash, Kimi K3.1 and AMD MI355X hardware.Open Models · via vLLM · 1d ago1 minMigrating vLLM to the LibTorch Stable ABIExplains how the vLLM project moved its CUDA build and compiled extensions to PyTorch's Stable ABI. The write-up covers what changed in the build and why this lets vLLM work with newer PyTorch releases without a full rebuild.Open Models · via vLLM · 1d ago1 minIntroducing the New Auto and Auto Code RoutersExplains the new Auto Router, rebuilt on a fresh technical foundation, and Auto Code, a router tuned for coding tasks. Both choose a model and reasoning effort for each request by weighing eleven factors, including live pricing, provider health, benchmarks, and cache state, within the models your governance policies permit.AI Coding Tools · via OpenRouter · 1d ago1 minHow Snorkel runs realistic agent simulations with ModalLearn how Snorkel uses Modal to run realistic agent simulations at scale. The article covers how Modal handles inference for the open-weight models Snorkel evaluates, and how its GPU and CPU sandboxes support millions of simulations each month.Agents · via Modal · 1d ago1 minConfigure, Certify, and Operate Kubernetes GPU Clusters with NVIDIA DSX OSExplains how to configure, certify, and operate Kubernetes GPU clusters with NVIDIA DSX OS, including why a single underperforming GPU can slow a distributed training job even when it passes basic health checks. Covers keeping clusters validated as nodes are replaced and drivers are patched.Observability · via NVIDIA Technical Blog · 2d ago1 minFive Ways NVIDIA Co-Design Maximizes the Value of Every Byte in the AI FactoryExplains how NVIDIA's co-design approach gets more useful work out of high bandwidth memory in AI factories, covering the costs of redundant weight copies, idle KV caches, and recomputed prefixes. Describes five specific strategies and the generational performance data behind them.Open Models · via NVIDIA Technical Blog · 2d ago1 minLe Chonk brings the brains. Studio puts it to work.Learn what Mistral Studio offers for building and deploying enterprise agents, workflows, and apps alongside the Le Chonk model. The piece covers how teams can keep control over their models, data, and infrastructure while moving to production.Agents · via Mistral · 2d ago1 min