Items tagged with “GPU”.
NVIDIA HGX B200 lowers embodied carbon intensity by 24% vs. HGX H100, while delivering higher AI performance and energy efficiency. This article reviews the PCF-backed improvements, new hardware features, and implications for developers and enterprises.
NVIDIA Dynamo offloads KV Cache from GPU memory to cost-efficient storage, enabling longer context windows, higher concurrency, and lower inference costs for large-scale LLMs and generative AI workloads.
A detailed look at seven battle-tested techniques used by Kaggle Grandmasters to solve large tabular datasets fast with GPU acceleration, from diversified baselines to advanced ensembling and pseudo-labeling.
Microsoft unveils plans for a 1.2 million-square-foot Fairwater AI data center in Wisconsin, housing hundreds of thousands of Nvidia GB200 GPUs. The project promises unprecedented AI training power with a closed-loop cooling system and a cost of $3.3 billion.
A detailed look at speculative decoding for AI inference, including draft-target and EAGLE-3 methods, how they reduce latency, and how to deploy on NVIDIA GPUs with TensorRT.
RAPIDS 25.08 introduces a function- and line-level profiler for cuml.accel, a default streaming executor for the Polars GPU engine, expanded datatype and string support, a new Spectral Embedding algorithm in cuML, and zero-code-change accelerations for several estimators.
An end-to-end look at deploying OpenAI GPT OSS models on SageMaker AI and Bedrock AgentCore to power a multi-agent stock analyzer with LangGraph, including 4-bit MXFP4 quantization, serverless orchestration, and scalable inference.
Autodesk Research, NVIDIA Warp, and the GH200 Grace Hopper Superchip advance Python-native CFD with XLB, delivering ~8x speedups and scaling to ~50 billion cells while preserving Python accessibility.
A detailed look at how NVIDIA Run:ai Model Streamer lowers cold-start times for LLM inference by streaming weights into GPU memory, with benchmarks across GP3, IO2, and S3 storage.
PyNvVideoCodec 2.0 brings major enhancements to Python-based GPU-accelerated video encoding, decoding, and transcoding, built on NVIDIA's Video Codec SDK and available via PyPI and NVIDIA NGC. This update combines C++-level speed with Python flexibility.
OpenAI, NVIDIA, and Nscale announce Stargate UK, a sovereign AI infrastructure partnership delivering local compute power in the UK to support public services, regulated industries, and national AI goals.
CUDA-accelerated VC-6 enables selective, multi-resolution decoding tailored for GPUs, dramatically reducing I/O and accelerating data-to-tensor paths in vision AI pipelines.
NVIDIA’s RTX PRO 6000 Blackwell Server Edition dramatically speeds protein-structure inference, enabling end-to-end GPU-resident workflows with OpenFold and TensorRT—achieving up to 138x faster folding than AlphaFold2.
CUDA is being redistributed by major third-party platforms to embed directly in package feeds, simplifying installation and dependency management for GPU-enabled applications and speeding deployment across ecosystems.
NVIDIA NIM Operator 3.0.0 expands scalable AI inference on Kubernetes, enabling multi-LLM and multi-node deployments, KServe integration, and DRA support in technology preview, with Red Hat collaboration and NeMo Guardrails.
Explores how NVIDIA Rivermax and NEIO FastSocket deliver ultra-low latency, high-throughput networking for financial services, leveraging kernel bypass, zero-copy, and GPUDirect to move data directly from NICs to GPUs.
A report on using PyTorch Ahead-of-Time (AoT) compilation to accelerate ZeroGPU Spaces on Nvidia H200 hardware, including setup, quantization options, dynamic shapes, and practical speedups.
A new GPU memory swap (model hot-swapping) approach lets multiple models share GPUs beyond capacity, reducing costs while preserving responsiveness for large-scale LLM inference.
A deep dive into nvMatmulHeuristics, a CUDA GPU kernel meta-parameter optimizer that accelerates GEMM auto-tuning with CUTLASS 4.2, delivering near-optimal kernels in far less time.
Learn how PyTorch AoT compilation speeds up ZeroGPU Spaces on Nvidia H200 GPUs, enabling instant reloads, FP8 quantization, dynamic shapes, and practical speedups (1.3x–1.8x) for Flux, Wan, and LTX models.
CUDA Toolkit 13.0 for Jetson Thor unifies ARM toolchains, enables Unified Virtual Memory, advanced GPU sharing, OpenRM/dmabuf interoperability, NUMA support, and enhanced developer tools for edge AI workloads.
Unified CUDA toolkit for Arm on Jetson Thor with full memory coherence, multi-process GPU sharing, OpenRM/dmabuf interoperability, NUMA support, and better tooling across embedded and server-class targets.
Leverage GPU memory swap (model hot-swapping) to share GPUs across multiple LLMs, reduce idle GPU costs, and improve autoscaling while meeting SLAs.
Introduces nvMatmulHeuristics to quickly select a small set of high-potential GEMM kernel configurations for CUTLASS 4.2, drastically reducing auto-tuning time while approaching exhaustive-search performance.
Learn how PyTorch AoT compilation speeds up ZeroGPU Spaces by exporting a compiled model once and reloading instantly, with FP8 quantization, dynamic shapes, and careful integration with the Spaces GPU workflow.
Microsoft’s AI division unveils its first homegrown models, MAI-Voice-1 and MAI-1-preview, signaling a move toward consumer-focused in-house AI capabilities and complementary tools for Copilot.
A detailed look at how NVIDIA scaled a LangGraph-based AI-Q research agent from a single user to hundreds, using NeMo Agent Toolkit, load testing, and phased rollout with observability.
Jetson Thor delivers edge-scale AI, enabling fast, multi-model generative reasoning with a Blackwell GPU, MIG, FP4/FP8, and 128 GB memory for next-gen robotic platforms.
A detailed guide to identifying and addressing five frequent pandas bottlenecks, with CPU-focused tips and a GPU-powered drop-in accelerator that can dramatically speed up workloads.
NVIDIA Blackwell Ultra accelerates AI training and inference with dual-die integration, NVFP4 4-bit precision, 288 GB HBM3e, and high-bandwidth interconnects, enabling scalable AI factories at scale.
NVIDIA outlines how Blackwell architecture, CUDA-X libraries, and a thriving open-source ecosystem—from RAPIDS to Cosmos and Nemotron—accelerate AI development from prototype to production.
OpenAI reorganizes leadership with Fidji Simo named CEO of Applications to oversee most staff, while Sam Altman shifts toward mega-scale compute, GPUs, brain-computer interfaces, and consumer hardware. A BCI startup will sit outside OpenAI.
A developer-focused resource outlining five common pandas bottlenecks, practical CPU and GPU fixes, and drop-in GPU acceleration with cudf.pandas for scalable data workflows.
An in‑depth profile of NVIDIA Blackwell Ultra, its dual‑die NV‑HBI design, NVFP4 precision, 288 GB HBM3e per GPU, and system‑level interconnects powering AI factories and large‑scale inference.
Overview of how NVIDIA's Blackwell hardware, open source models, and a rich software stack accelerate AI—from data prep to deployment—through broad model/tool availability and end-to-end infrastructure.
cuPQC v0.4 adds expanded hash function support and Merkle-tree calculations, accelerating data integrity checks and security for high‑speed cryptographic workloads on GPUs.
NVIDIA HPC SDK v25.7 advances unified memory programming for GPU-accelerated HPC, automating CPU-GPU data movement and boosting developer productivity on coherent architectures like Grace Hopper.
NVIDIA explores how NVLink Fusion extends scale-up NVLink technologies to hyperscalers, enabling custom CPUs and XPUs with a rack-scale fabric for AI inference at scale.
Learn how Code Editor and multiple spaces in Amazon SageMaker Unified Studio accelerate end-to-end ML workflows, from development to training, evaluation, and optional deployment.
cuPQC v0.4 expands hash function support and adds Merkle-tree calculations to accelerate data integrity, verification, and privacy-preserving cryptography on GPUs.
Overview of NVLink Fusion, enabling scalable, high-bandwidth AI inference across custom CPU/XPU configurations and rack-scale NVLink fabrics.
NVIDIA HPC SDK v25.7 advances unified memory programming to automate CPU-GPU data movement, boost developer productivity, and simplify porting of ocean models like NEMO to GPUs on Grace Hopper–based systems.
Learn how to deploy and stream Omniverse Kit apps at scale across cloud and on-prem environments with flexible options, templates, and real-world deployments.
NVIDIA NeMo-RL v0.3 adds Megatron-Core support to boost training throughput for large models, addressing DTensor limitations, with long-context training, MoE support, and simplified configuration.
Guidance for deploying and streaming Omniverse Kit Apps at scale with flexible options across cloud, on-prem, and DGX Cloud.
Overview of NeMo-RL v0.3 with Megatron-Core backend for post-training large models, detailing 6D/4D parallelism, GPU-optimized kernels, and simplified configuration to boost reinforcement learning throughput on models at scale.
Open Nemotron Nano 2 9B delivers leading accuracy and up to 6x throughput with a Hybrid Transformer–Mamba backbone and a configurable thinking budget, aimed at edge, PC and enterprise AI agents.
A step-by-step guide to authoring, building, versioning, and publishing production-ready CUDA kernels using the Hugging Face kernel-builder and reproducible Nix builds.
A practical guide to developing, building for multiple architectures, and deploying CUDA kernels with Hugging Face Kernel Builder. Learn how to create a robust workflow from local development to Hub-based distribution.
A practical walkthrough of Hugging Face's kernel-builder for developing, compiling, and deploying production-ready CUDA kernels across PyTorch, with reproducible builds, multi-arch support, and hub-based distribution.
Salesforce and AWS collaborated to optimize GPU utilization, boost resource efficiency, and cut costs by deploying multiple foundation models on SageMaker AI endpoints using inference components. This approach enables granular per-model resource control and dynamic scaling.
Dynamo 0.4 introduces disaggregated serving, SLO-based autoscaling, AIConfigurator, and enhanced observability to accelerate large-model inference at scale with improved efficiency.
NVIDIA and partners unveil Wheel Variants, a new format to tailor Python wheels to exact hardware, enabling smoother CUDA-accelerated package installs and scalable packaging workflows.
NVIDIA announces general access for Isaac Sim 5.0 and Isaac Lab 2.2 at SIGGRAPH 2025, delivering new robot models, enhanced sensor simulation, ROS 2 integration, and cloud-based deployment via NVIDIA Brev.
Overview of Accelerate's ND‑Parallel capabilities: combining data, tensor, sharded and context parallelism with Axolotl to scale large-model training across multi‑GPU and multi‑node environments.
A concise resource on composing multiple parallelism strategies (DP, FSDP, TP, CP) with Accelerate and Axolotl to train large models across many GPUs, with guidance on configuration, use cases, and trade‑offs.
cuDF introduces JIT-compiled transforms to fuse kernels, reduce intermediates, and boost ETL throughput on GPUs. The 25.08 release adds ternary and string operators, with trade-offs around compilation time and kernel caching.
XGBoost 3.0 enables terabyte-scale GBDT training on a single GH200 Grace Hopper Superchip, streaming data from host RAM via external-memory Quantile DMatrix for faster, simpler scaling.
CUDA Toolkit 13.0 introduces tile-based programming, unified Arm tooling across servers and embedded devices, and notable improvements in Nsight Compute, fatbin compression, and deployment workflows—positioning CUDA for future GPUs and Tensor Core architectures.
NVIDIA and OpenAI optimize gpt-oss-120b and gpt-oss-20b for accelerated inference on Blackwell, achieving up to 1.5M tokens per second on GB200 NVL72, with Day 0 support across cloud to edge.
OpenAI unveils gpt-oss-120b and gpt-oss-20b—two open-weight LLMs designed for strong real-world performance at low cost. Licensed under Apache 2.0, they emphasize reasoning, tool use, and efficient on-device deployment across consumer hardware.
NVIDIA vGPU 19.0 takes full advantage of the RTX PRO Blackwell architecture, enabling MIG-enabled graphics and AI virtualization with up to 48 concurrent VMs per GPU and new tools for AI workflows in virtualized data centers.
OpenAI releases two open-weight, mixture-of-experts models—GPT OSS 120B and GPT OSS 20B—utilizing MXFP4 4-bit quantization. They’re licensed Apache 2.0 with a minimal usage policy and are accessible via Hugging Face Inference Providers for on-device, on-prem, and cloud deployments.
OpenAI releases GPT OSS, a pair of 120B and 20B mixture-of-experts models with MXFP4 4-bit quantization, licensed Apache 2.0, and delivered via Hugging Face Inference Providers for on-device and server deployments.
Two open-weight OpenAI models (gpt-oss-120b and gpt-oss-20b) using mixture-of-experts and MXFP4 quantization for fast, memory-efficient reasoning. Licensed Apache 2.0; designed for private/local deployments and on-device use, with Hugging Face Inference Providers integration.
xT enables end-to-end modeling of gigapixel-scale images on modern GPUs using nested tokenization, region encoders, and long-context vision, delivering high fidelity and context on images up to 29,000×25,000 pixels.
End-to-end modeling of extremely large images on contemporary GPUs via nested tokenization and region/context encoders, delivering richer context with lower memory footprints.