Nvidia GB300
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 12
- How to Serve GLM-5.3 Across Two NVIDIA DGX Stations
NVIDIA Technical Blog shows how to serve the quantized 753-billion-parameter GLM-5.3 locally with vLLM across two connected NVIDIA DGX Stations. Tensor parallelism splits weights and computation between the GB300 GPUs, keeping the roughly 464 GB checkpoint in GPU memory behind one OpenAI-compatible API. In NVIDIA’s SPEED-Bench test, throughput peaked at 607.8 tokens per second at concurrency 16 before falling to 385.0 tokens per second at concurrency 32.
NVIDIA Technical Blog · 🔥 6 - vLLM-Omni adds day-zero serving for MiniMax H3-Image
The vLLM Blog reports that vLLM-Omni now supports MiniMax H3-Image, an open-weight model for native high-resolution text-to-image generation and image editing. On four NVIDIA Vera Rubin GPUs, the team reports a 0.87-second median for native 4096×4096 generation in a short-prompt serving benchmark. Separate tests report an average of 1.48 seconds for 4K generation on four GB300 GPUs and 1.77 seconds for a 4K edit with one reference. A detailed prompt test on Rubin averaged 1.09 seconds. Implementations and recipes also cover AMD Instinct, Intel Arc Pro, Huawei Ascend and Moore Threads hardware, with validation configurations and results varying by platform.
vLLM Blog · 🔥 0 - LMSYS Reports Muse Spark Open Decoding Above 1,000 Tokens per Second
LMSYS says it built a megakernel from scratch in CUDA C++ and PTX for Muse Spark Open, running each layer’s attention and MoE work as one kernel with extensive latency hiding. On eight GB300 GPUs, the reported decoding throughput increased from 517 to 1,109 tokens per second. The implementation supports speculative decoding but currently only batch size one.
LMSYS Blog · 🔥 0 - Microsoft Rejects Nscale Texas Capacity Expansion Option
Microsoft reportedly declined an option to more than double its Texas compute-capacity contract with Nscale, forgoing additional capacity that the Financial Times said could have been worth tens of billions of dollars. Nscale clarified that the option covered about 500 MW at its own Texas site, while the existing contract remains in place and the company reportedly seeks a valuation of up to $35 billion for its IPO.
36氪快讯 · 🔥 0 - Analysis: Power, Not Chips, Could Cap AI Compute at About 80 GW in 2027
Tomasz Tunguz argues that the binding constraint on AI in 2027 will be energized compute rather than chip supply, projecting a ceiling of roughly 80 GW of powered AI capacity. He cites GPU short-term rental rates rising from about $3 to $12-13 per hour in six months, Anthropic's blended GPU-hour cost up 82% in two quarters per its S-1/A, and finished server racks sitting unpowered because data centers lack energized utility power. Brad Gerstner said on an All-In livestream that 10 to 15 GW of the expected 2027 build could fail to come online, and Tunguz estimates the current 38 GW fleet earns about $140 billion annually versus roughly $500 billion combined with new capacity.
Tomasz Tunguz · 🔥 0 - Deutsche Bank sees SpaceX AI compute at 2.7GW by year-end
Deutsche Bank forecasts that SpaceX's AI computing capacity could reach about 2.7GW by year-end as additional NVIDIA GB300 chips are deployed, within the company's 2.5GW to 3.0GW target range. The research note, relayed by 36Kr and Cailian Press, says a cluster of 220,000 GB300 chips represents roughly 440MW and could support cloud-compute contracts and Grok inference demand. It also anticipates more large cloud-compute contracts or expanded customer orders and maintains a Buy rating with a $250 target price.
36氪快讯 · 🔥 0 - Baseten cuts GPU-resident delta weight sync time to 15 seconds
Baseten redesigned its delta weight-sync path to use model weights already resident on GPUs as the parent copy, eliminating the need for a second full checkpoint in CPU memory. The company reports that syncing an approximately 770 GB GLM-5.3 checkpoint now takes 15 seconds with a 0.8-second request pause, versus 36 seconds and six seconds previously. Baseten also reports up to 3.2× higher output throughput per GPU at concurrency 48, while its Kimi K3 tests include an 8.4 GB delta applied across two GB300 nodes.
Baseten Blog · 🔥 0 - SGLang Reports Matching or Beating DeepInfer
DeepSeek open-sourced DeepInfer, the inference engine behind its own service, along with related kernels. In project-run tests using DeepInfer’s benchmarks and the same hardware, SGLang reached up to 1.04x prefill throughput on four GB300 or four B300 GPUs and cut decode step time by as much as 7.4% on 64 GB300 GPUs, according to the SGLang report. A 36氪 report said vLLM also approached or partly exceeded DeepInfer under its disclosed configurations. The comparisons used project-reported results and differing workloads and measurement methods.
LMSYS Blog · 🔥 0 - AWS stockpiles NVIDIA GB300 racks as power constraints bite
Business Insider, citing Amazon internal emails and documents, reported that AWS expected to receive about $5 billion in NVIDIA GB300 racks in August 2026 and roughly $3 billion more each month afterward. Some racks are being stored because data centers lack enough power, while an early-ownership process has pushed payments ahead of deployment. Amazon said staging completed hardware before power is ready is standard industry practice.
智东西 · 🔥 0 - Microsoft pulls out of Nscale's UK data center as grid power slips to the 2030s
Nscale said Microsoft has withdrawn from the Loughton, Essex data center the British neocloud firm was building for it, after UK grid authorities told Nscale not to expect a power supply until the 2030s; the site had been designed to host 23,040 Nvidia GB300 GPUs, starting at 50 MW and expandable to 90 MW. Microsoft was the only publicly announced customer, and Nscale says it remains committed to the project, is exploring alternative power sources and is looking at other UK expansion sites while calling its Microsoft relationship strong and long-term. The exit comes as Nscale seeks to raise as much as $3 billion in a US IPO, with Microsoft and Anthropic accounting for about 85% of its contracts, and Bloomberg reported investors pushing for a valuation of $16–19 billion against the roughly $30 billion Nscale had sought.
The Information · 🔥 0 - Blog SGLang Powers Beam in Production Reflection’s Beam is a 501B-parameter mixture-of-experts model with 23B active parameters per token, built for software engineering and long-horizon agentic tasks. SGLang Team October 16, 2026
On this page Highlights Model Architecture Hybrid Attention and Output Gating Dense Layers and Grouped MoE Routing SSMax Sandwich Normalization and Depth-Scaled Residuals Feature Support Dflash2 Speculative Decoding w/ Data-Parallel Attention Unified Memory for Target and Draft KV Heterogeneous TP for Prefill/Decode Disaggregation Reasoning, Tool Calls and LoRA BF16, NVFP4 and FP8 Model Support Prefix Reuse, Chunked Prefill and HiCache Optimizations Fusion Optimizations BF16 MoE Kernel Optimization TPU Support Reinforcement learning in Miles Validated Run Performance Result
LMSYS Blog · 🔥 0 - TokenSpeed reaches 685 output tokens/s per user on Qwen3.8-Flash-Next
Qwen's official X account congratulated LightSeek Foundation after its TokenSpeed inference engine reached 685 output tokens per second per user on Qwen3.8-Flash-Next. The run used 4× NVIDIA GB300 GPUs with NVFP4, and LightSeek published a blog post on the kernel and serving optimizations behind the result. Qwen said it is excited to see the community unlock more of Qwen's inference potential; the figure comes from LightSeek's own benchmark and has not been independently verified.
Qwen (X) · 🔥 0
Experience and discussion from the community
Share my Nvidia GB300 experienceAsk about Nvidia GB300
Nobody has shared their experience with Nvidia GB300 yet.