SA Article Coverage Review · 2026-03-24_nvidia-the-inference-kingdom-expands

已有新版 Atomic Claims

本頁保留原畫線、人工筆記及歷史 Claim。最新正式資料請開啟 Phase C–E 全文 Coverage；審查結果見完成報告。

Coverage Summary

Source: 開啟原始 SA 文章
Atomic Claims: 382
Source blocks: 232
Blocks with ≥1 Atomic Claim: 126
Blocks without Atomic Claim: 106
Unplaced Claims: 0

Coverage Review

請從頭到尾閱讀下方 SA 全文。Atomic Claim 會依 Evidence 在原文出現的位置 inline 插入。
繁中翻譯只作為閱讀輔助顯示；不會進入 source、Claim provenance 或 Graph。
沒有 Claim callout 的段落不一定有問題；若內容重要且應形成知識，請記到 Missing Claim Notes。

Missing Claim Notes

若看到重要但沒有 Atomic Claim 的段落，請在這裡記錄：
- Section:
- Evidence:
- 為什麼重要／應該抽成什麼 Claim：

SA Full Text + Translation + Atomic Claims

Nvidia – The Inference Kingdom Expands

Source: Nvidia

At GTC 2026, Nvidia delivered an event packed full of ground breaking announcements. Nvidia’s pace of innovation is not showing any signs of slowing, as they introduced three entirely new systems this year: Groq LPX, Vera ETL256, and STX. Also announced were updates to Nvidia’s Kyber rack architecture system, CPO making its debut for scale-up networking with the unveiling of the Rubin Ultra NVL576 and Feynman NVL1152 multi-rack systems. Early hints on Feynman’s architecture was also a key topic. A Jensen callout for InferenceX during the keynote was a highlight.

GTC 2026，Nvidia 帶來一場塞滿重大 announcement 的發表會。Innovation pace 完全沒有放慢，今年一次推出三套全新 system：Groq LPX、Vera ETL256、STX。同時還公布 Kyber rack architecture 更新，並讓 CPO 首次進入 scale-up networking，推出 Rubin Ultra NVL576 與 Feynman NVL1152 multi-rack system；Feynman architecture 的早期線索也是重點之一。Jensen 在 keynote 上特別點名 InferenceX，則是另一個亮點 ↗。

Atomic Claim 1/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0001

Claim: 在 GTC 2026，Nvidia 舉行了一場充滿重大新公告的活動。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 2/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0002

Claim: Nvidia 的創新速度沒有放緩跡象，今年一次推出三套全新 system：Groq LPX、Vera ETL256 與 STX。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 3/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0003

Claim: 同場也公布 Nvidia Kyber rack architecture 更新；CPO 隨 Rubin Ultra NVL576 與 Feynman NVL1152 multi-rack system 揭露，首次導入 scale-up networking。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 4/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0004

Claim: Feynman architecture 的早期線索也是此次內容的重要主題。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 5/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0005

Claim: Jensen 在 keynote 中特別點名 InferenceX，被原文列為一項 highlight。
Frame: ATTRIBUTE · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟逐條審核

This is our GTC 2026 recap, and we will address many of the key questions that have been left unanswered by Nvidia. Specifically, we will go through the LPX rack and LP30 chip and explain how attention and feed forward network disaggregation (AFD) works; more details on the various rack architectures behind NVL144, NVL576, and NVL1152 and clarify just how much optics will be inserted as well as the rationale behind the dense Vera ETL256.

這是我們的 GTC 2026 recap，接下來會回答許多 Nvidia 尚未說明的關鍵問題。內容包括 LPX rack 與 LP30 chip，以及 attention 與 feed-forward network disaggregation（AFD）如何運作；也會進一步說明 NVL144、NVL576、NVL1152 背後不同 rack architecture、實際會導入多少 optics，以及高密度 Vera ETL256 的設計理由。

Atomic Claim 6/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0006

Claim: 本文是 SemiAnalysis 的 GTC 2026 recap。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 7/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0007

Claim: SemiAnalysis 表示，本文將處理許多 Nvidia 尚未回答的重要問題。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 8/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0008

Claim: 具體而言，本文會先介紹 LPX rack……
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 9/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0009

Claim: ……以及 LP30 chip，並解釋 attention 與 feed-forward network disaggregation 如何運作。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 10/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0010

Claim: 本文也會提供 NVL144、NVL576 與 NVL1152 背後 rack architecture 的更多細節，說明 optics 導入程度，以及高密度 Vera ETL256 的設計 rationale。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

The next generation Kyber rack had some big updates and some hidden details.

這篇是我們的 GTC 2026 recap，也會回答許多 Nvidia 留下的關鍵問題。具體來說，我們會拆解 LPX rack、LP30 chip，說明 attention 與 feed-forward network disaggregation（AFD）如何運作；進一步說明 NVL144、NVL576、NVL1152 背後不同 rack architecture，到底會插入多少 optics，以及 dense Vera ETL256 的設計邏輯。下一代 Kyber rack 也有幾項重大更新，以及一些 Nvidia 沒明說的細節。

Atomic Claim 11/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0011

Claim: 下一代 Kyber rack 有多項重大更新，也包含一些尚未明示的細節。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Groq

First up is the Groq LPU. One of the most significant recent events in AI infrastructure was Nvidia’s “acquisition” of Groq. Strictly speaking, Nvidia paid Groq $20B to license their IP and hire most the team. This functions almost as an acquisition, though its structure technically falls short of it being legally considered as one, thereby simplifying or obviating the need for regulatory approvals.

首先是 Groq LPU。近期 AI infrastructure 最重要的事件之一，是 Nvidia 對 Groq 的「acquisition」。嚴格來說，Nvidia 支付 Groq 200 億美元取得 IP license，並聘用其大部分團隊；實質效果接近 acquisition，但法律結構並不構成正式收購，因此可簡化、甚至避免部分 regulatory approval。

Atomic Claim 12/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0012

Claim: 接下來首先討論 Groq LPU。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 13/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0013

Claim: AI 基礎設施近期最重大的事件之一，是 Nvidia 對 Groq 的「收購」。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 14/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0014

Claim: 嚴格來說，Nvidia 向 Groq 支付 200 億美元，以取得其 IP 授權並聘用大部分團隊成員。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 15/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0015

Claim: 這項交易的實質效果幾乎等同收購，但其結構在法律上尚不足以被視為正式收購，因此可簡化、甚至免除監管核准需求。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Given Nvidia’s market share, if this transaction were structured as a full acquisition and were put to anti-trust review, such a transaction would likely not go through. The other benefit is that it avoids a drawn-out transaction closing process. Nvidia got instant access to Groq’s IP and people. This is why, less than four months after the deal was announced, Nvidia already has a system concept that is being integrated into the Vera Rubin inference stack.

先從 Groq LPU 開始。近期 AI infrastructure 最重要事件之一，就是 Nvidia 對 Groq 的『acquisition』。嚴格來說，Nvidia 支付 Groq $20B，是為了 license Groq IP 並雇用大部分團隊。實質效果幾乎等於收購，但法律結構刻意沒有做到完整 acquisition，因此可簡化、甚至避開 regulatory approval。考量 Nvidia market share，如果交易真的以完整收購形式送 anti-trust review，很可能根本過不了。另一個好處是避免漫長 closing process；Nvidia 幾乎立即取得 Groq IP 與人員。這也解釋了為什麼交易宣布不到四個月，Nvidia 就已經拿出一個 system concept，準備整合進 Vera Rubin inference stack。

Atomic Claim 16/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0016

Claim: 考量 Nvidia 的市場占有率，若這項交易被設計為完整收購，
Frame: CLAIM_ONLY · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 17/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0017

Claim: 若接受反壟斷審查，這類交易很可能無法通過。
Frame: CLAIM_ONLY · Mode: EXPECTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 18/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0018

Claim: 另一項好處是，這種交易結構可避免冗長的交易交割流程。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 19/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0019

Claim: Nvidia 立即取得了 Groq 的 IP 與人才。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 20/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0020

Claim: 因此，在交易宣布不到四個月後，Nvidia 已經提出一個正整合進 Vera Rubin inference stack 的系統概念。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Let’s now go through a refresher on the LPU architecture to see how Groq’s LPU complements Nvidia’s GPU. For more details see our original Groq piece. ↗ The premise from that piece remains unchanged: the standalone Groq LPU system is not economical for serving tokens at scale, but it can serve tokens very quickly which can demand a large market premium. This is the premise behind how LPU fits into a disaggregated decode system.

接著快速複習 LPU architecture，看 Groq LPU 如何和 Nvidia GPU 互補。更多細節可參考我們原本的 Groq 文章 ↗。當時的核心論點完全沒變：standalone Groq LPU system 若拿來大規模 serving token，economics 並不好；但它可以把 token 產生得非常快，因此有機會收取很高 market premium。這正是 LPU 被放進 disaggregated decode system 的基本邏輯。

Atomic Claim 21/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0021

Claim: 接下來先回顧 LPU 架構，以了解 Groq 的 LPU 如何補足 Nvidia 的 GPU。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 22/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0022

Claim: 若要查看更多細節，可參考我們先前關於 Groq 的文章。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 23/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0023

Claim: 先前文章的核心看法仍未改變：獨立運作的 Groq LPU 系統，在大規模 token serving 下並不具經濟性。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 24/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0024

Claim: Groq LPU 可以非常快速地提供 tokens，因此可能取得很高的市場溢價。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 25/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0025

Claim: 這也是 LPU 能夠納入 disaggregated decode 系統的基本前提。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

LPU chip

Groq’s first and only publicly announced LPU architecture was detailed in their ISCA 2020 paper. Unlike typical hardware architectures connecting many general-purpose cores, Groq re-organized the architecture into groups of single-purpose units connecting to other groups of different purposes, and they named the groups “slices.” Between functional units are streaming registers, scratchpad SRAM for functional units to pass data to each other.

Groq 第一個、也是唯一曾公開公布的 LPU architecture，詳述於其 ISCA 2020 paper。不同於一般硬體把許多 general-purpose core 互連，Groq 將架構重組成不同用途的 single-purpose unit group，並把這些 group 稱為「slice」。各 functional unit 之間透過 streaming register 相連，並使用 scratchpad SRAM 讓 functional unit 彼此傳遞資料。

Atomic Claim 26/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0026

Claim: Groq 第一個、也是唯一公開宣布過的 LPU 架構，詳述於其 ISCA 2020 論文。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 27/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0027

Claim: 不同於把許多通用核心連接起來的典型硬體架構，Groq 將架構重組為不同用途的單一功能單元群組，並把這些群組稱為「slices」；功能單元之間配置 streaming registers 與 scratchpad SRAM，讓功能單元彼此傳遞資料。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Groq opted for single-level scratchpad SRAM instead of multi-level memory hierarchy to make the hardware execution deterministic.

Groq 第一代、也是唯一公開完整介紹過的 LPU architecture，來自 ISCA 2020 paper。和一般把很多 general-purpose core 互連的 hardware architecture 不同，Groq 把 architecture 重新組成多組 single-purpose unit，再讓不同用途的 group 互相連接，並把這些 group 稱為『slice』。Functional unit 之間使用 streaming register，也就是 scratchpad SRAM，讓不同 unit 交換 data。Groq 刻意採 single-level scratchpad SRAM，而不是 multi-level memory hierarchy，目的是讓 hardware execution 具有 deterministic 特性。

Atomic Claim 28/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0028

Claim: Groq 選擇單層 scratchpad SRAM，而不是多層記憶體階層，以讓硬體執行具備 deterministic 特性。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Concretely, LPU architecture has VXM slices for vector operations, MEM slices for loading/storing data, SXM slices for tensor shape manipulation, and MXM slices for performing matrix multiplication. Spatially, the slices are laid out horizontally, allowing the data to stream horizontally. Within a slice, instructions are pumped vertically across units. Conceptually, LPU resembles a systolic array that pumps instructions vertically and data horizontally.

具體來說，LPU architecture 有負責 vector operation 的 VXM slice、load／store data 的 MEM slice、tensor shape manipulation 的 SXM slice，以及執行 matrix multiplication 的 MXM slice。空間布局上，這些 slice 水平排列，data 因此沿水平方向 streaming；每個 slice 內 instruction 則垂直穿過各 unit。概念上，LPU 很像一個 instruction 垂直流動、data 水平流動的 systolic array。

Atomic Claim 29/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0029

Claim: 具體而言，LPU 架構包含：負責 vector operations 的 VXM slices、負責資料 load/store 的 MEM slices、負責 tensor shape manipulation 的 SXM slices，以及負責 matrix multiplication 的 MXM slices。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 30/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0030

Claim: 在空間配置上，這些 slices 以水平方向排列，使資料能沿水平方向串流。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 31/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0031

Claim: 在單一 slice 內，instructions 會跨越各單元沿垂直方向傳送。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 32/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0032

Claim: 概念上，LPU 類似 systolic array：instructions 沿垂直方向傳送，而資料沿水平方向流動。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Source: Groq, SemiAnalysis

The data flow and instruction flow design requires fine-grained pipelining to achieve high performance. Since LPU architecture makes computation deterministic, the compiler can aggressively schedule and overlap instructions to hide latency. The LPU’s use of high bandwidth SRAM and aggressive pipelining are the two main factors that enable LPU’s low latency.

這種 data flow、instruction flow design 要靠非常 fine-grained pipelining 才能得到高 performance。因為 LPU architecture 讓 computation 變成 deterministic，compiler 可以非常 aggressive 地 schedule、overlap instruction，把 latency 隱藏起來。High-bandwidth SRAM + aggressive pipelining，就是 LPU low latency 的兩大核心來源。

Atomic Claim 33/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0033

Claim: 這種資料流與 instruction flow 設計需要 fine-grained pipelining，才能達到高效能。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 34/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0034

Claim: 由於 LPU 架構讓運算具備 deterministic 特性，compiler 可以更積極地排程並重疊 instructions，以隱藏 latency。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 35/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0035

Claim: LPU 使用高頻寬 SRAM。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 36/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0036

Claim: 積極的 pipelining 也是促成 LPU 低 latency 的兩大主要因素之一。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

LPU gen 1 was designed on a legacy Global Foundries 14nm process, with Marvell responsible for the chip’s physical design. This was a much more mature node compared to peers when it taped out in 2020, with the incumbent AI chip platforms mostly on TSMC’s N7 platform. This made sense for an early product focused on proving out Groq’s architecture and bringing its inference-centric design to market.

LPU Gen 1 採用較舊的 GlobalFoundries 14nm process，由 Marvell 負責 chip physical design。它在 2020 年 tape out 時，相較同業多數 AI chip platform 使用的 TSMC N7，製程成熟許多。對早期產品而言這是合理選擇，因為重點是先驗證 Groq architecture，並把 inference-centric design 推向市場。

Atomic Claim 37/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0037

Claim: 第一代 LPU 採用較舊的 Global Foundries 14nm 製程設計。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 38/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0038

Claim: Marvell 負責這顆晶片的 physical design。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 39/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0039

Claim: 這個節點在 2020 年 tapeout 時，相較同業已成熟許多；當時主流 AI 晶片平台大多採用 TSMC 的 N7 平台。
Frame: COMPARISON · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 40/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0040

Claim: 對一款主要用來驗證 Groq 架構的早期產品而言，這樣的選擇是合理的，
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 41/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0041

Claim: 並把以 inference 為核心的設計推向市場。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

The 14nm node was mature, relatively well understood, and suitable for an initial chip where architectural differentiation mattered more than pushing its silicon to the leading edge.

LPU Gen1 採較舊的 GlobalFoundries 14nm，由 Marvell 負責 chip physical design。2020 tapeout 時，主流 AI chip platform 大多已在 TSMC N7，因此 14nm 明顯是成熟很多的 node。不過對 Groq 早期產品來說這是合理選擇：重點是先證明 architecture、把 inference-centric design 推到市場，而不是硬上 leading-edge silicon。14nm 成熟、理解充分，對一顆 architecture differentiation 比 transistor scaling 更重要的 first chip 很適合。

Atomic Claim 42/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0042

Claim: 14nm 製程已相當成熟、業界也較熟悉，因此適合初代晶片；對這類產品而言，架構差異化比把 silicon 推到最先進節點更重要。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

One of the selling points is that the chip can be manufactured and packaged entirely in the United States compared to their competitors being heavily reliant on the Asia semiconductor supply chain: logic and packaging in Taiwan, with HBM from Korea.

它的一個賣點是 chip 可以完全在美國製造、封裝；競爭對手則高度依賴亞洲 semiconductor supply chain：logic、packaging 在台灣，HBM 來自韓國。

Atomic Claim 43/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0043

Claim: 其中一項賣點是，這顆晶片可以在美國境內完成製造，
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 44/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0044

Claim: 並可完全在美國完成封裝；相較之下，競爭對手高度依賴亞洲半導體供應鏈，包括台灣的 logic 與 packaging，以及韓國供應的 HBM。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Since then, Groq’s roadmap has stalled due to execution, with no LPU 2 having been shipped. This leaves the Groq LPU looking even more dated against competing roadmaps. What was once a meaningful but still manageable node disadvantage versus 7nm-era peers has widened into a far sharper gap, with all leading accelerator platforms now moving onto 3nm-class processes in 2026.

但後來 Groq roadmap 因 execution 卡住，LPU 2 一直沒有真正出貨，讓 Groq LPU 相較競爭 roadmap 看起來越來越老。原本對 7nm-era peer 而言還只是明顯、但可控制的 node disadvantage，到 2026 年所有 leading accelerator platform 都轉向 3nm-class process 後，差距已經拉得非常大。

Atomic Claim 45/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0045

Claim: 此後，Groq 的 roadmap 因執行問題而陷入停滯。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 46/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0046

Claim: LPU 2 至今沒有實際出貨。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 47/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0047

Claim: 這也使 Groq LPU 相較競爭對手的 roadmap 顯得更加落後。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 48/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0048

Claim: 原本雖然明顯、但仍可控制的，
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 49/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0049

Claim: 相較 7nm 世代同業的製程節點劣勢，如今已擴大成更尖銳的差距；到 2026 年，所有領先的 accelerator 平台都正轉向 3nm 等級製程。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

The follow on Groq LPU 2 was designed for Samsung Foundry’s SF4X node, specifically at Samsung’s Austin fab, allowing them to extend the pitch that Groq is fabricated domestically in the USA. Samsung would also provide support for the back-end design. The choice of Samsung was driven by favorable terms / investment, with Samsung Foundry struggling to find customers for its advanced nodes and missing out on an AI logic customer.

後續 Groq LPU 2 原本規劃採用 Samsung Foundry SF4X node，並在 Samsung Austin fab 生產，讓 Groq 可以延續「美國本土製造」的訴求。Samsung 也會提供 back-end design 支援。選擇 Samsung 的原因包括較有利的條件／投資；當時 Samsung Foundry 在 advanced node 上難以找到客戶，也缺少 AI logic customer。

Atomic Claim 50/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0050

Claim: 後續的 Groq LPU 2 是為 Samsung Foundry 的 SF4X 節點設計，具體是在 Samsung Austin fab 生產，讓 Groq 得以延續「在美國國內製造」的訴求。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 51/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0051

Claim: Samsung 也會為 back-end design 提供支援。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 52/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0052

Claim: 選擇 Samsung 的原因，是其提供有利的條件／投資；同時 Samsung Foundry 當時正苦於替先進製程節點尋找客戶，且尚未拿到 AI logic 客戶。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Unsurprisingly, Samsung was a key investor in Groq’s subsequent Series D in August 2024, and most recently in September 2025 before the Nvidia “acquisition.”

後續 Groq LPU 2 原本設計給 Samsung Foundry SF4X，且指定 Samsung Austin fab，讓 Groq 可以延續『美國本土製造』的 pitch。Samsung 也會支援 back-end design。選 Samsung 的原因是條件與投資都很優惠；Samsung Foundry 當時先進 node 缺 customer，也一直沒拿到 AI logic 大客戶。毫不意外，Samsung 後來成為 Groq 2024 年 8 月 Series D 的重要 investor，2025 年 9 月、Nvidia『acquisition』前也再次投資。

Atomic Claim 53/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0053

Claim: Samsung 隨後成為 Groq 2024 年 8 月 Series D 的重要投資人，並在 2025 年 9 月、Nvidia「收購」前再次投資。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

However, the Groq LPU 2 was never productized because of design issues. The C2C SerDes on the chip couldn’t hit the advertised 112G speed which caused the design to malfunction, as we detailed long ago in the Accelerator model. ↗ The third generation Groq LPU is the one that Nvidia will be productizing.

但 Groq LPU 2 最後因 design issue 從未 productize。Chip 上的 C2C SerDes 達不到 advertised 112G speed，導致整體 design 無法正常運作；我們很早以前就在 Accelerator Model ↗ 詳細寫過。Nvidia 最後真正會 productize 的，是第三代 Groq LPU。

Atomic Claim 54/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0054

Claim: 不過，Groq LPU 2 因設計問題，最終從未產品化。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 55/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0055

Claim: 晶片上的 C2C SerDes 無法達到宣稱的 112G 速度，導致設計無法正常運作。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 56/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0056

Claim: 第三代 Groq LPU 將由 Nvidia 推動產品化。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

SRAM and Memory Hierarchy

We have written about the role of SRAM in the memory hierarchy, but the quick recap is that SRAM is very fast (low latency and high bandwidth) but this comes at the expense of density and therefore cost.

我們之前已寫過 SRAM 在 memory hierarchy 的角色；簡單回顧就是：SRAM 非常快，latency 低、bandwidth 高，但代價是 density 低，因此成本很高。

Atomic Claim 57/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0057

Claim: 我們過去已討論過 SRAM 在 memory hierarchy 中的角色。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 58/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0058

Claim: 簡要來說，SRAM 速度非常快，具備低 latency 與高 bandwidth，但代價是較低的 density，因而帶來較高成本。
Frame: ATTRIBUTE · Mode: INFERRED · Mapping: COMPLETE
開啟逐條審核

SRAM machines such as Groq’s LPU therefore enable very fast time to first token and tokens per second per user but at the expense of total throughput, as their limited SRAM capacity quickly gets saturated by weights, with little left over for KVcache that grows as more users are batched. GPUs win for throughput and cost as we have shown.

因此，Groq LPU 這類 SRAM machine 能提供很快的 time to first token 與每位 user 的 tokens per second，但代價是 total throughput 較低。有限的 SRAM capacity 很快就會被 weight 佔滿，剩餘空間不足以容納會隨 batching user 增加而成長的 KV cache。就 throughput 與 cost 而言，GPU 仍然占優勢，正如我們先前分析所示。

Atomic Claim 59/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0059

Claim: 因此，像 Groq LPU 這類以 SRAM 為核心的機器，可以提供很快的 time to first token，以及很高的每位使用者 tokens per second。
Frame: ATTRIBUTE · Mode: INFERRED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 60/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0060

Claim: 但代價是 total throughput 較低，因為有限的 SRAM 容量很快會被模型 weights 填滿，剩餘可供 KVcache 使用的空間不多，而 KVcache 會隨 batch 中使用者增加而成長。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 61/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0061

Claim: 如我們先前所示，GPU 在 throughput 與成本方面更具優勢。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

This is why Nvidia has decided to combine these architectures to get the best of both worlds: accelerate parts of decode that are more latency sensitive and are not as memory heavy on a low-latency SRAM-heavy chip like the LPU, while memory hungry attention is performed on GPUs that come with a lot of fast (but not SRAM fast) memory capacity.

因此 Groq LPU 這類 SRAM machine 可以做到很快的 time-to-first-token 與 tokens/sec/user，但 total throughput 會受限。原因是 SRAM capacity 很快就被 model weight 塞滿，留給 KV cache 的空間很少，而 batch user 越多，KV cache 又會持續增加。正如我們已展示，GPU 在 throughput、cost 上更有優勢。這也解釋 Nvidia 為什麼選擇把兩種 architecture 結合：decode 中對 latency 更敏感、memory footprint 沒那麼大的部分，交給低 latency、SRAM-heavy 的 LPU；memory-hungry attention 則由 GPU 處理，因為 GPU 有大量高速 memory capacity，雖然還是沒有 SRAM 那麼快。

Atomic Claim 62/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0062

Claim: 因此 Nvidia 決定結合兩種架構，以加速 decode 中對延遲較敏感的部分。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 63/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0063

Claim: 這些部分對 memory 的需求較低，適合放在像 LPU 這種低 latency、以 SRAM 為主的晶片上執行。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 64/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0064

Claim: memory-intensive 的 attention 則由 GPU 執行；GPU 擁有大量高速記憶體容量，雖然速度仍不及 SRAM。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Source: SemiAnalysis

This brings us to the Groq 3 LPU or LP30, with LPU gen 2 being skipped over. This chip has no Nvidia design involvement. The SerDes issues affecting v2 appear to be fixed. Behind the paywall, we will reveal the SerDes IP vendor which may come as a surprise. Nvidia also announced an LP35 which is a minor refresh of the LP30 which will remain on SF4 and will require a new tapeout.

接下來是 Groq 3 LPU，也就是 LP30，中間的 LPU Gen 2 被跳過。這顆 chip 沒有 Nvidia 參與設計；影響 v2 的 SerDes 問題看起來已經修正。付費內容中我們會揭露 SerDes IP vendor，結果可能令人意外。Nvidia 也公布 LP35，屬於 LP30 的小幅 refresh，仍採 SF4，但需要一次新的 tapeout。

Atomic Claim 65/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0065

Claim: 接著是 Groq 第三代 LPU，也就是 LP30；第二代 LPU 被直接跳過。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 66/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0066

Claim: 這顆晶片的設計沒有 Nvidia 參與。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 67/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0067

Claim: 影響 v2 的 SerDes 問題看起來已經修復。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 68/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0068

Claim: 在付費內容中，我們將揭露 SerDes IP 供應商，而這個答案可能令人意外。
Frame: CLAIM_ONLY · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 69/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0069

Claim: Nvidia 也宣布 LP35；它是 LP30 的小幅更新版，並將繼續採用 SF4。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 70/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0070

Claim: 它需要一次新的 tapeout。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

It will incorporate NVFP4 number format but given Nvidia is prioritizing time to market we don’t expect any other drastic design changes.

這就帶到 Groq 3 LPU，也就是 LP30；LPU Gen2 會直接跳過。這顆 chip 沒有 Nvidia design involvement，而影響 v2 的 SerDes issue 看起來已經解決。Paywall 後我們會揭露 SerDes IP vendor，答案可能會讓人意外。Nvidia 也公布 LP35，這是 LP30 的 minor refresh，仍採 SF4，但需要新的 tapeout；它會加入 NVFP4 number format。不過 Nvidia 現在最重視 time-to-market，因此我們不預期其他 design 會有很劇烈的變化。

Atomic Claim 71/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0071

Claim: LP35 將支援 NVFP4 數值格式。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 72/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0072

Claim: 考量 Nvidia 正優先追求 time to market，我們不預期還會有其他大幅度的設計變更。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Source: Nvidia

LPU 3’s near reticle size die layout is very similar to LPU 1. a significant amount of area taken is up by the 500MB of on-chip SRAM, with a very small amount of area dedicated to MatMul cores that offer 1.2 PFLOPs of FP8 compute – a fraction of compute compared to Nvidia GPUs. This compares to LPU 1 with 230MB of SRAM and 750 TFLOPs of INT8, with the performance increase mostly driven by node migration from GF16 to SF4. As a single monolithic die, advanced packaging isn’t required.

LPU 3 的 near-reticle-size die layout 和 LPU 1 非常相似。Die 上很大一部分面積都被 500MB on-chip SRAM 占用，真正分給 MatMul core 的 area 很小，只提供 1.2 PFLOPs FP8 compute，和 Nvidia GPU 相比只是零頭。LPU 1 則有 230MB SRAM、750 TFLOPs INT8；這次 performance improvement 主要來自製程從 GF16 移到 SF4。因為仍是 single monolithic die，所以不需要 advanced packaging。

Atomic Claim 73/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0073

Claim: LPU 3 接近 reticle size 的 die layout 與 LPU 1 非常相似；其中相當大的 die area 被 500MB on-chip SRAM 占據。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 74/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0074

Claim: 只有很小一部分 die area 配置給 MatMul cores，提供 1.2 PFLOPs 的 FP8 compute；相較 Nvidia GPU，其運算能力僅是一小部分。
Frame: COMPARISON · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 75/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0075

Claim: 相比之下，LPU 1 配備 230MB SRAM，並提供 750 TFLOPs 的 INT8 運算能力。
Frame: COMPARISON · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 76/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0076

Claim: 效能提升主要來自製程節點從 GF16 遷移至 SF4。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 77/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0077

Claim: 由於採用單一 monolithic die，因此不需要先進 packaging。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

One of the benefits of relying on SF4 is that it isn’t constrained like TSMC’s N3, which is putting a cap on accelerator production and is a key reason why the industry remains compute constrained. This is in addition to not having HBM which is also constrained. This allows Nvidia to ramp production of the LPU without sacrificing or eating into their valuable TSMC allocation or HBM allocations, representing true incremental revenue and capacity that noone else can access.

採 SF4 的一個優點，是不像 TSMC N3 那樣受 capacity constraint。N3 capacity 正在限制 accelerator production，也是整個產業持續 compute constrained 的重要原因 ↗。LPU 同時也不使用另一個供應受限的 HBM ↗。因此 Nvidia 可以 ramp LPU production，而不會犧牲或侵蝕自己珍貴的 TSMC allocation、HBM allocation，形成真正 incremental 的 revenue、capacity，而且這些 capacity 其他 competitor 無法取得。

Atomic Claim 78/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0078

Claim: 採用 SF4 的一項優勢，是它不像 TSMC N3 那樣受產能限制，而 N3 的限制正壓抑加速器產量。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 79/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0079

Claim: 這也是整個產業仍受到 compute constraint 的關鍵原因之一。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 80/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0080

Claim: 此外，這套方案也不需要使用同樣受到供給限制的 HBM。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 81/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0081

Claim: 這讓 Nvidia 可以擴大 LPU 產量，而不必犧牲或侵蝕珍貴的 TSMC 產能配額與 HBM 配額，因此形成真正的 incremental revenue。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 82/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0082

Claim: 這也代表一塊其他人無法取得的額外產能。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Since Nvidia has taken over, the next generation LP40 will be fabricated on TSMC N3P and use CoWoS-R, and Nvidia will contribute more of their own IP such as supporting the NVLink protocol rather than Groq’s C2C. This will be the first LPU to be extremely co-designed alongside the Feynman platform. Groq’s original plans for LPU Gen 4 was also with TSMC and Alchip as the back-end design partner. Alchip’s involvement is now redundant with Nvidia able to perform backend design on their own.

Nvidia 接手之後，下一代 LP40 將採 TSMC N3P 與 CoWoS-R，並導入更多 Nvidia 自有 IP，例如改支援 NVLink protocol，而不是 Groq 的 C2C。這會是第一顆與 Feynman platform 進行 extreme co-design 的 LPU。Groq 原本的 LPU Gen 4 計畫同樣採 TSMC，並由 Alchip 擔任 back-end design partner；現在 Nvidia 已能自行做 backend design，Alchip 的角色因此不再必要。

Atomic Claim 83/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0083

Claim: 自 Nvidia 接手後，下一代 LP40 將採用 TSMC N3P 製程。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 84/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0084

Claim: 它將使用 CoWoS-R；Nvidia 也會投入更多自有 IP，例如支援 NVLink protocol，而不是沿用 Groq 的 C2C。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 85/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0085

Claim: 這將是第一個與 Feynman 平台進行高度共同設計的 LPU。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 86/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0086

Claim: Groq 原先對 LPU Gen 4 的規劃，同樣是與 TSMC 合作。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 87/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0087

Claim: Alchip 擔任 back-end design partner。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 88/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0088

Claim: 由於 Nvidia 已能自行執行 backend design，Alchip 的參與如今變得多餘。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

One of the technical innovations planned is hybrid bonded DRAM to extend on-chip memory with only a slight decrease in latency and bandwidth vs SRAM, but much higher performance compared to DRAM. SK Hynix was tapped as the supplier of the DRAM to be used for the 3D stacking. All of this and more was detailed long ago in the Accelerator model. ↗

Nvidia 接手後，下一代 LP40 會改用 TSMC N3P、CoWoS-R，而且 Nvidia 會加入更多自家 IP，例如改支援 NVLink protocol，而不是沿用 Groq C2C。這也會是第一代和 Feynman platform 進行 extreme co-design 的 LPU。Groq 原本的 LPU Gen4 計畫同樣是 TSMC 製程，back-end design partner 是 Alchip；現在 Nvidia 自己就能做 backend design，因此 Alchip 的角色已變得多餘。規劃中的一項 technical innovation，是使用 hybrid-bonded DRAM 擴充 on-chip memory；相較 SRAM，latency、bandwidth 只小幅下降，但比一般 DRAM 快很多。SK Hynix 原本被選為 3D stacking DRAM supplier。這些細節我們很早以前就在 Accelerator Model ↗ 說過。

Atomic Claim 89/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0089

Claim: 規劃中的一項技術創新是採用 hybrid-bonded DRAM 擴充 on-chip memory；相較 SRAM，其延遲與頻寬只略為下降，但相較一般 DRAM 具更高效能。
Frame: COMPARISON · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 90/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0090

Claim: SK Hynix 被選為 3D stacking 所使用 DRAM 的供應商。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 91/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0091

Claim: 上述內容以及更多細節，早已在 Accelerator model 中說明。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Source: Nvidia, SemiAnalysis Accelerator Model

GPU and LPU Integration: Attention FFN Disaggregation (AFD)

Source: Nvidia

Now with an understanding of what LPUs are good for we can understand how they fit into inference setups. NVIDIA introduced LPUs to improve the performance of high interactivity scenarios. In those scenarios, LPUs can leverage their low-latency capabilities to improve the decode phase latencies. One way LPUs can improve decode phase latencies is by applying the Attention FFN Disaggregation (AFD) technique, introduced in MegaScale-Infer and Step-3.

理解 LPU 的強項後，就能看它如何放進 inference setup。NVIDIA 導入 LPU，主要是為了改善 high-interactivity scenario 的 performance；這類情境可以利用 LPU low-latency 特性，降低 decode phase latency。其中一種方式，就是使用 MegaScale-Infer ↗、Step-3 ↗ 提出的 Attention-FFN Disaggregation（AFD）。

Atomic Claim 92/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0092

Claim: 在理解 LPU 擅長的工作後，就能進一步理解它們如何應用於 inference setup。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 93/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0093

Claim: NVIDIA 導入 LPU，以提升高互動情境下的效能。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 94/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0094

Claim: 在這類情境下，LPU 可利用低延遲能力，改善 decode 階段的延遲。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 95/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0095

Claim: LPU 改善 decode 階段延遲的一種方式，是採用 Attention FFN Disaggregation（AFD）；此技術曾在 MegaScale-Infer 與 Step-3 中提出。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

As we explained in our InferenceX article, LLM inference involves two phases: prefill and decode. Prefill processes the full input context: It is compute-intensive, which is suitable for GPUs. On the other hand, decode predicts new tokens and is memory-bounded. Decode is latency-sensitive because the model predicts new tokens one by one, and LPU’s high SRAM bandwidth and low-latency capabilities can help accelerate this iterative process.

如我們 InferenceX 文章 ↗ 所解釋，LLM inference 有兩個 phase：prefill、decode。Prefill 會處理完整 input context，compute intensity 高，因此適合 GPU；decode 則是一個 token 一個 token 預測，屬 memory-bound、又對 latency 很敏感。LPU 的 high SRAM bandwidth、low latency，正好可以幫助加速這個 iterative decode process。

Atomic Claim 96/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0096

Claim: 如 InferenceX 文章所述，LLM inference 包含兩個階段：prefill 與 decode。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 97/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0097

Claim: Prefill 會處理完整 input context；它屬於 compute-intensive 工作，因此適合由 GPU 執行。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 98/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0098

Claim: 另一方面，decode 會預測新的 token，且屬於 memory-bounded 工作。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 99/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0099

Claim: Decode 對延遲非常敏感，因為模型必須逐一預測新的 token。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 100/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0100

Claim: LPU 的高 SRAM 頻寬與低延遲能力，有助於加速這個反覆迭代的過程。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Source: SemiAnalysis

Attention and FFN are subsets of operations in a model. In a model forward pass, attention’s output feeds into a token router, and the token router assigns each token to k experts, where each expert is an FFN. Attention and FFN have very different performance properties. During decode phase, the GPU utilization of attention barely improves when scaling batch size due to being bounded by loading KV cache. In contrast, the GPU utilization of FFN scales with batch size comparatively better.

Attention、FFN 都是 model 中的 operation subset。一次 forward pass 裡，attention output 會送進 token router，由 router 把每個 token 指派給 top-k expert，而每個 expert 本質上就是一個 FFN。Attention、FFN 的 performance property 很不一樣：decode phase 裡，attention 因為受 KV cache loading 限制，就算增加 batch size，GPU utilization 也幾乎不會改善；FFN 則相對會隨 batch size 增加而更有效率。

Atomic Claim 101/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0101

Claim: Attention 與 FFN 都是模型中的 operation subset。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 102/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0102

Claim: 關於 AFD：在模型 forward pass 中，attention 的輸出會送入 token router。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 103/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0103

Claim: token router 會將每個 token 分配給 k 個 expert，而每個 expert 都是一個 FFN；Attention 與 FFN 的效能特性差異很大。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 104/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0104

Claim: 在 decode 階段，attention 的 GPU utilization 在提高 batch size 時幾乎不會改善，因為受到載入 KV cache 的限制。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 105/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0105

Claim: 相較之下，GPU 執行 FFN 時的 utilization，隨 batch size 增加而改善的幅度相對更好。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

This is something we have worked with certain hardware vendors and memory companies on with our inference simulator for more than 6 months.

這也是過去超過六個月，我們用 inference simulator 和部分 hardware vendor、memory company 一直共同研究的主題 ↗。

Atomic Claim 106/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0106

Claim: SemiAnalysis 表示，他們已透過 inference simulator 與部分 hardware vendor 及 memory company 共同研究這個問題超過 6 個月。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Source: MegaScale-Infer, SemiAnalysis

As state-of-the-art mixture-of-expert (MoE) models grow increasingly sparse, tokens can choose experts from a larger expert pool. As a result, each expert receives fewer tokens, leading to lower utilization. This motivates attention and FFN disaggregation. If a GPU only performs attention operations, its HBM capacity can be fully allocated to KV cache, increasing the total number of tokens it can process, which then increases the tokens each expert processes on average.

隨 SOTA MoE model 越來越 sparse，token 可以從更大的 expert pool 中選擇，結果每個 expert 平均收到的 token 變少，utilization 下降。這就是 Attention、FFN disaggregation 的動機。如果某顆 GPU 只負責 attention，整個 HBM capacity 都能分給 KV cache，提高它可同時處理的 token 數量；token 總量增加後，每個 expert 平均能分到的 token 也會增加。

Atomic Claim 107/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0107

Claim: 隨著 state-of-the-art Mixture-of-Experts 模型變得愈來愈 sparse，token 可以從更大的 expert pool 中選擇 expert。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 108/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0108

Claim: 因此，每個 expert 分到的 token 變少，導致 utilization 降低。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 109/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0109

Claim: 這促成了 attention 與 FFN 的 disaggregation。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 110/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0110

Claim: 如果 GPU 只執行 attention operation，其 HBM 容量就能全部分配給 KV cache，提高可處理的 token 總數，進而增加每個 expert 平均處理的 token 數。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟逐條審核

Source: SemiAnalysis

Comparing the two operations, we see attention is stateful due to dynamic KV cache loading patterns, whereas FFN is stateless since the computation only depends on the token inputs. Thus, we disaggregate the computation of attention and FFN. We map attention computations to GPUs, which handle dynamic workloads well. For FFNs, we map them to LPUs, since LPU architecture is inherently deterministic and benefits from static compute workloads.

比較兩種 operation，attention 是 stateful，因為 KV cache loading pattern 會動態變化；FFN 則是 stateless，computation 只取決於 token input。因此可以把 attention、FFN computation 拆開：attention map 到善於處理 dynamic workload 的 GPU；FFN 則 map 到 LPU，因為 LPU architecture 本質上 deterministic，更適合 static compute workload。

Atomic Claim 111/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0111

Claim: 比較兩種運算後，SemiAnalysis 認為 attention 因為有動態 KV cache 載入模式，因此屬於 stateful 工作負載。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 112/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0112

Claim: FFN 是 stateless，因為其運算只依賴 token input。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 113/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0113

Claim: 因此，attention 與 FFN 的運算會被拆分。
Frame: CLAIM_ONLY · Mode: INFERRED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 114/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0114

Claim: attention computation 會被分配到 GPU，因為 GPU 擅長處理 dynamic workload。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 115/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0115

Claim: FFN 會被分配到 LPU，因為 LPU 架構本質上是 deterministic，並能受益於 static compute workload。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Source: SemiAnalysis, MegaScale-Infer

With AFD, token routing from GPUs to LPUs can become the bottleneck, especially under strict latency constraints. The token routing flow involves two operations: dispatch and combine. In the dispatch step, we route each token to their top k experts with an All-to-All collective operation. After experts complete their computation, we perform the combine step, where the outputs are sent back to the source location with a reverse All-to-All collective, continuing the next layer’s computation.

AFD 下，GPU 到 LPU 的 token routing 可能變成 bottleneck，尤其 latency constraint 很嚴時。Token routing 有兩個 operation：dispatch、combine。Dispatch 階段用 All-to-All collective，把每個 token route 到 top-k expert；expert 計算完成後，combine 再用 reverse All-to-All，把 output 傳回 source location，接著繼續下一層 computation。

Atomic Claim 116/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0116

Claim: 在 AFD 下，從 GPU 到 LPU 的 token routing 可能成為 bottleneck，尤其是在嚴格 latency constraint 下。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 117/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0117

Claim: token routing 流程包含兩個 operation：dispatch 與 combine。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 118/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0118

Claim: 在 dispatch 階段，每個 token 會透過 All-to-All collective operation 被送到其 top-k experts。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 119/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0119

Claim: expert 完成運算後會進入 combine 階段，輸出透過 reverse All-to-All collective 傳回來源位置，接著繼續下一層運算。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Source: SemiAnalysis

To hide the communication latency of dispatch and combine, we employ ping pong pipeline parallelism. In addition to splitting batches into micro-batches and computation pipelining like standard pipeline parallelism, the tokens dispatched to the LPUs are combined back to the source GPUs, so they ping pong between the GPUs and the LPUs.

為了隱藏 dispatch、combine 的 communication latency，可以使用 ping-pong pipeline parallelism。除了像一般 pipeline parallelism 一樣，把 batch 切成 micro-batch 並做 computation pipelining，送到 LPU 的 token 計算完後還要 combine 回 source GPU，因此 token 會在 GPU、LPU 之間來回 ping-pong。

Atomic Claim 120/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0120

Claim: 為了隱藏 dispatch 與 combine 的 communication latency，系統採用 ping-pong pipeline parallelism。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 121/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0121

Claim: 除了像標準 pipeline parallelism 一樣把 batch 切成 micro-batch 並做 computation pipelining，送到 LPU 的 token 還會被 combine 回來源 GPU，因此 token 會在 GPU 與 LPU 之間來回 ping-pong。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Source: MegaScale-Infer

Source: SemiAnalysis

Source: MegaScale-Infer

Source: SemiAnalysis

Speculative Decoding

A different way LPUs could improve decode phase latencies is by accelerating a speculative decoding setup, where we deploy draft models or Multi-Token Prediction (MTP) layers onto LPUs.

另一種利用 LPU 降低 decode phase latency 的方式，是加速 speculative decoding：把 draft model 或 Multi-Token Prediction（MTP）layer 部署在 LPU 上。

Atomic Claim 122/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0122

Claim: LPU 改善 decode 階段延遲的另一種方式，是加速 speculative decoding setup，把 draft model 或 Multi-Token Prediction（MTP） layer 部署到 LPU 上。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟逐條審核

For a decoding step of context N tokens, adding k additional tokens during forward pass (a warm prefill of k new tokens) marginally increases the latency when k << N. Using this property, speculative decoding uses a small draft model or MTP layers to predict k new tokens, saving time since small models have lower latency per decode step. To verify the draft tokens, the main model only needs one warm prefill of k new tokens, at the latency cost of roughly a single decode step.

在 context 有 N 個 token 的 decoding step 中，若 forward pass 新增 k 個 token（也就是對 k 個新 token 做 warm prefill），當 k 遠小於 N 時，latency 只會小幅增加。Speculative decoding 利用這個特性，由較小的 draft model 或 MTP layer 預測 k 個新 token；小模型每個 decode step latency 較低，因此可以節省時間。主模型驗證 draft token 時，只需要做一次 k-token warm prefill，latency cost 大致相當於單次 decode step。

Atomic Claim 123/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0123

Claim: 對 context 為 N 個 token 的 decoding step，在 forward pass 中額外加入 k 個 token（即對 k 個新 token 做 warm prefill）時，只要 k 遠小於 N，增加的 latency 就很有限。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 124/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0124

Claim: 利用上述特性，speculative decoding 會使用較小的 draft model 或 MTP layer 預測 k 個新 token；因小模型每個 decode step 的 latency 較低，因此可以節省時間。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 125/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0125

Claim: 為了驗證 draft token，main model 只需要對 k 個新 token 做一次 warm prefill，其 latency cost 約等於一次 decode step。
Frame: COMPARISON · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Speculative decoding usually boosts output token per decode step by 1.5 to 2 tokens, depending on the draft model / MTP accuracy. With its low latency capabilities, LPUs can further increase the latency savings and improve throughput.

對 context 已有 N token 的 decode step 而言，只要 k << N，在 forward pass 額外加入 k 個 token，也就是做一次 k-token warm prefill，latency 增量其實很小。Speculative decoding 就利用這個特性：先讓小型 draft model 或 MTP layer 預測 k 個新 token；因 small model 每 decode step latency 更低，所以能節省時間。驗證這些 draft token 時，main model 只需要做一次 k-token warm prefill，latency cost 大致接近單一步 decode。依 draft model／MTP accuracy 不同，speculative decoding 通常可以把每個 decode step 的 output token 數提高到約 1.5–2 個。LPU latency 更低，因此還能進一步放大這些 latency saving、提高 throughput。

Atomic Claim 126/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0126

Claim: Speculative decoding 通常能把每個 decode step 的 output token 提升到 1.5～2 個，實際幅度取決於 draft model／MTP 的準確率。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 127/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0127

Claim: 憑藉低 latency 能力，LPU 可進一步提高 latency savings 並改善 throughput。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Source: SemiAnalysis

For LPUs, deploying a draft model or MTP layers is quite different from applying AFD. FFNs are stateless, while draft models and MTP layers require dynamic KV cache loading. Each FFN is around hundreds of megabytes, whereas draft models and MTP layers take up tens of gigabytes. To support this memory usage, LPUs can access up to 256 GB of DDR5 per Fabric Expansion Logic FPGAs on the LPX compute tray.

對 LPU 來說，部署 draft model／MTP layer 和做 AFD 很不一樣。FFN 是 stateless；draft model、MTP layer 則需要動態載入 KV cache。單個 FFN 通常只有數百 MB，但 draft model、MTP layer 會吃到數十 GB。為支援這些 memory usage，LPX compute tray 上的每顆 Fabric Expansion Logic FPGA 都可 access 最多 256GB DDR5。

Atomic Claim 128/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0128

Claim: 對 LPU 而言，部署 draft model 或 MTP layer，和套用 AFD 有很大差異。
Frame: COMPARISON · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 129/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0129

Claim: FFN 是 stateless，而 draft model……
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 130/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0130

Claim: MTP layer 需要動態載入 KV cache。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 131/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0131

Claim: 每個 FFN 的容量約為數百 MB。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 132/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0132

Claim: draft model 與 MTP layer 會占用數十 GB 的容量。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 133/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0133

Claim: 為了支援這種 memory usage，LPU 可透過 LPX compute tray 上每顆 Fabric Expansion Logic FPGA 存取最高 256GB 的 DDR5。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

LPX Rack System

Let’s look at the LPX rack system, which has interesting details. Nvidia has displayed an LPX rack with 32 1U LPU compute trays with 2 Spectrum-X switches. This 32 tray 1U version that Nvidia has shown off at GTC is very close to Groq’s original server design before the acquisition. We believe that this server configuration is not the version that will be shipped in 3Q, with Nvidia implementing changes. Here, we will detail what we know about the actual production version. This was already detailed in the Accelerator model. ↗

接著看 LPX rack system，裡面有一些很有意思的細節。Nvidia 在 GTC 展示的是一個 32 個 1U LPU compute tray + 2 台 Spectrum-X switch 的 LPX rack。這個 32-tray、1U 版本其實非常接近 Groq 被收購前原本的 server design。我們認為這不是第三季真正會出貨的 production version，Nvidia 還會做修改。下面會整理我們目前知道的實際 production version；這些資訊先前已在 Accelerator Model ↗ 提過。

Atomic Claim 134/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0134

Claim: 接下來看 LPX rack system，其中有一些值得注意的細節。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 135/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0135

Claim: Nvidia 展示過一套 LPX rack，配置 32 個 1U LPU compute tray 與 2 個 Spectrum-X switch。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 136/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0136

Claim: Nvidia 在 GTC 展示的這個 32-tray、1U 版本，與 Groq 在收購前的原始 server design 非常接近。
Frame: COMPARISON · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 137/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0137

Claim: SemiAnalysis 認為這個 server configuration 並不是 3Q 會出貨的版本，Nvidia 仍會進行修改。
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 138/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0138

Claim: 接下來將說明目前已知的實際 production version。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 139/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0139

Claim: 這些內容先前已在 Accelerator model 中說明。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

LPX Compute Tray

Each LPX compute tray or node has 16 LPUs with 2 Altera FPGAs, 1 Intel Granite Rapids host CPU and 1 BlueField-4 front-end module. As with other Nvidia systems, hyperscalers customers can and will use their own Front-end NIC of choice rather than paying for Nvidia’s BlueField.

每個 LPX compute tray／node 內含 16 顆 LPU、2 顆 Altera FPGA、1 顆 Intel Granite Rapids host CPU，以及 1 個 BlueField-4 frontend module。和其他 Nvidia system 一樣，hyperscaler customer 可以、也會使用自己選擇的 frontend NIC，而不是額外付費買 Nvidia BlueField。

Atomic Claim 140/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0140

Claim: 每個 LPX compute tray／node 配置 16 顆 LPU、2 顆 Altera FPGA、1 顆 Intel Granite Rapids host CPU，以及 1 個 BlueField-4 front-end module。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 141/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0141

Claim: 和其他 Nvidia 系統相同，hyperscalers 客戶可以自行選擇前端網路方案。
Frame: COMPARISON · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 142/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0142

Claim: 會使用自己選擇的 Front-end NIC，而不是支付 Nvidia 的 BlueField。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

The LPU modules are mounted in a belly-to-belly on the PCB, meaning 8 LP30 modules on the top side of the PCB and the other 8 LP30 modules on the bottom. All of the connectivity that comes out of the LPU are via PCB traces and given the dense all-to-all mesh for intra-node connections this requires a very high spec PCB to support the routing. The belly-to-belly mounting is used to reduce PCB trace lengths across the ‘X’ and ‘Y’ dimensions.

LPU module 採 belly-to-belly 方式裝在 PCB：8 個 LP30 module 在 PCB 正面，另外 8 個在背面。LPU 所有 connectivity 都走 PCB trace；又因 intra-node 是非常密集的 all-to-all mesh，routing 對 PCB specification 要求非常高。Belly-to-belly mounting 的目的，就是縮短 X、Y 維度上的 PCB trace length。

Atomic Claim 143/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0143

Claim: LPU module 在 PCB 上採 belly-to-belly 安裝：8 個 LP30 module 位於 PCB 上側，另外 8 個位於下側。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 144/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0144

Claim: LPU 對外的所有 connectivity 都透過 PCB trace。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 145/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0145

Claim: 由於 intra-node connection 採 dense all-to-all mesh，需要非常高規格的 PCB 才能支援 routing。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 146/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0146

Claim: belly-to-belly mounting 用來縮短 PCB 在 X 與 Y 維度上的 trace length。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Something interesting about the system is the important role the FPGAs play. Nvidia refers to the FPGAs as “Fabric Expansion Logic” which serves multiple purposes. First, they act as a NIC which converts the LPU’s C2C protocol into Ethernet to connect to the Spectrum-X based ethernet scale-out fabric. It is this scale-out fabric through which the LPUs connect to GPUs in the decode system.

這套 system 很有意思的一點，是 FPGA 扮演非常重要的角色。Nvidia 把 FPGA 稱為『Fabric Expansion Logic』，用途很多。第一，它本身扮演 NIC，把 LPU 的 C2C protocol 轉成 Ethernet，接到 Spectrum-X-based Ethernet scale-out fabric。LPU 就是透過這套 scale-out fabric，連到 decode system 裡的 GPU。

Atomic Claim 147/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0147

Claim: 這套系統一個值得注意的地方，是 FPGA 扮演非常重要的角色。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 148/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0148

Claim: Nvidia 將這些 FPGA 稱為「Fabric Expansion Logic」，而且它們具有多種用途。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 149/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0149

Claim: 首先，這些 FPGA 會充當 NIC，把 LPU 的 C2C protocol 轉成 Ethernet，以連接到以 Spectrum-X 為基礎的 Ethernet scale-out fabric。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 150/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0150

Claim: LPU 就是透過這個 scale-out fabric，在 decode system 中連接到 GPU。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Second, the LPUs also traverse through the FPGAs to reach the host CPU, with the FPGAs converting C2C to PCIe to the CPU.

第二，LPU 要連 host CPU 也必須經過 FPGA，由 FPGA 把 C2C 轉成 PCIe，再連到 CPU。

Atomic Claim 151/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0151

Claim: 其次，LPU 也會經過 FPGA 才能連到 host CPU；FPGA 會把 C2C 轉成 PCIe 再連至 CPU。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Third, the FPGAs are connected to the backplane to talk to other FPGAs in the node, we believe this is to help manage control flow and timing of all the LPUs. The FPGAs also bring extra system DRAM of up to 256GB each. This pool of memory can be used for KVCache if the user wants the entire decode process served by the LPX.

第三，FPGA 也接到 backplane，和 node 內其他 FPGA 溝通；我們認為這主要用來協調所有 LPU 的 control flow、timing。每顆 FPGA 另外可帶最多 256GB system DRAM；如果使用者希望整個 decode process 都在 LPX 上完成，這池 memory 也可以拿來存 KV cache。

Atomic Claim 152/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0152

Claim: 第三，FPGA 會連到 backplane，以和 node 內其他 FPGA 溝通；SemiAnalysis 認為這可能是為了協助管理所有 LPU 的 control flow 與 timing。
Frame: NARY_RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 153/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0153

Claim: 每顆 FPGA 還可提供最高 256GB 的額外 system DRAM。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 154/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0154

Claim: 如果使用者希望整個 decode process 都由 LPX 提供服務，這個 memory pool 可以拿來存放 KVCache。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟逐條審核

On the front panel there are 8 x OSFP cages for cross-rack C2C, while there will be 2 cages (likely QSFP-DD) that goes to the Spectrum-switches that is used to connect the LPUs and the GPUs for the disaggregated decode system. We will share more about this when we describe the network.

Front panel 有 8 個 OSFP cage，供 cross-rack C2C 使用；另外還有 2 個 cage——很可能是 QSFP-DD——連到 Spectrum switch，用來把 LPU、GPU 接進 disaggregated decode system。等後面講 network 時會再展開。

Atomic Claim 155/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0155

Claim: 前面板配置 8 個 OSFP cages，用於跨機櫃 C2C 連線。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 156/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0156

Claim: 預計會有 2 個 cage（原文標示 likely §2§）連到 Spectrum switch，用來連接 LPU。
Frame: CLAIM_ONLY · Mode: EXPECTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 157/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0157

Claim: GPU 用於 disaggregated decode system。
Frame: RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 158/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0158

Claim: 後續在說明 network 時會再補充更多相關內容。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

LPU Network

The LPU network can be divided into the scale-up ‘C2C’ network and scale-out network which interacts with the Nvidia GPUs through Spectrum-X. First let’s discuss the scale-up network which can be divided into 3 portions: intra-node, inter-node/intra-rack, inter-rack. For C2C within the rack Nvidia announced a total of 640TB/s of scale up bandwidth per rack which comes from 256 LPUs x 90 lanes x 112Gbps/8 x 2 directions = 645TB/s.

LPU network 可分成 scale-up 的「C2C」network，以及透過 Spectrum-X 與 Nvidia GPU 互動的 scale-out network。先看 scale-up network，又可再分成 intra-node、inter-node／intra-rack、inter-rack 三部分。Nvidia 公布 rack 內 C2C scale-up bandwidth 合計為 640TB/s，計算來自 256 顆 LPU × 90 lanes × 112Gbps ÷ 8 × 雙向，約為 645TB/s。

Atomic Claim 159/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0159

Claim: LPU network 可以分成 scale-up 的 C2C network。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 160/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0160

Claim: scale-out network 會透過 Spectrum-X 與 Nvidia GPU 互動。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 161/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0161

Claim: scale-up network 可以分成 3 個部分：intra-node、inter-node／intra-rack、以及 inter-rack。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 162/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0162

Claim: 對 rack 內的 C2C，Nvidia 宣布每櫃總 scale-up bandwidth 為 640TB/s；依 256 顆 LPU × 90 lanes × 112Gbps ÷ 8 × 2 directions 計算約為 645TB/s。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Note that Nvidia uses the total 112G line rate rather than 100G of effective data rate.

LPU network 可以分成 scale-up『C2C』network，以及透過 Spectrum-X 和 Nvidia GPU 互動的 scale-out network。先看 scale-up，它又可分三段：intra-node、inter-node/intra-rack、inter-rack。Rack 內 C2C 部分，Nvidia 宣布每 rack scale-up bandwidth 總計 640TB/s，計算來自 256 LPUs × 90 lanes × 112Gbps ÷ 8 × 2 directions = 645TB/s。要注意，Nvidia 用的是完整 112G line rate，而不是約 100G effective data rate。

Atomic Claim 163/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0163

Claim: 要注意的是，Nvidia 採用總 112G line rate，而不是 100G effective data rate。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Intra-Tray Topology

Within each tray or node, all 16 LPUs are connected to each other in an all-to-all mesh. Each LPU module connects to the 15 other LPUs within the node with 4x100G of C2C bandwidth. Note that this ‘C2C’ is not related to NVLink, but Groq’s own scaleup fabric. These connections are all via PCB trace, which necessitates an extremely high spec PCB to support this routing density.

在每個 tray／node 內，16 顆 LPU 以 all-to-all mesh 彼此連接。每顆 LPU module 透過 4×100G C2C bandwidth，連到 node 內其他 15 顆 LPU。這裡的「C2C」與 NVLink 無關，而是 Groq 自有的 scale-up fabric。這些連線全部走 PCB trace，因此需要規格極高的 PCB 才能支援這種 routing density。

Atomic Claim 164/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0164

Claim: 每個 tray／node 內的 16 顆 LPU 彼此以 all-to-all mesh 連接。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 165/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0165

Claim: 每個 LPU module 會以 4×100G 的 C2C bandwidth，連接 node 內另外 15 顆 LPU。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 166/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0166

Claim: 這裡的 C2C 與 NVLink 無關，而是 Groq 自有的 scale-up fabric。
Frame: COMPARISON · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 167/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0167

Claim: 這些連線全部走 PCB trace，因此需要極高規格的 PCB 才能支撐這種 routing density。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

This is why the belly-to-belly layout is used: it reduces the ‘X’ and ‘Y’ distance between all the LPUs and instead have routing go in the ‘Z’ dimension.

每個 tray／node 內，16 顆 LPU 全部做 all-to-all mesh。每顆 LPU 用 4×100G C2C bandwidth 連到同 node 另外 15 顆 LPU。這裡的『C2C』和 NVLink 無關，是 Groq 自己的 scale-up fabric。所有 connection 都走 PCB trace，因此需要極高規格 PCB 才能承受 routing density。這也是採 belly-to-belly layout 的原因：縮短 LPU 間 X、Y 距離，讓 routing 更多改走 Z dimension。

Atomic Claim 168/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0168

Claim: 這也是使用 belly-to-belly layout 的原因：它會縮短 X……
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 169/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0169

Claim: ……所有 LPU 之間的 Y 距離，並改讓 routing 走 Z 維度。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

The LPU also has 1x100G going to one FPGA, with each FPGA interfacing with 8 LPUs. The 2 FPGAs each have 8x PCIe Gen 5 going to the CPUs. The LPU needs to traverse through the FPGA to interface with the CPU as LPUs don’t have PCIe PHYs to interface directly.

每顆 LPU 另外有 1×100G 連到一顆 FPGA，每顆 FPGA 負責 8 顆 LPU。兩顆 FPGA 各自再用 8×PCIe Gen5 連到 CPU。因為 LPU 本身沒有 PCIe PHY，無法直接和 CPU interface，所以一定得透過 FPGA。

Atomic Claim 170/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0170

Claim: LPU 另有 1×100G 連到一顆 FPGA，而每顆 FPGA 會介接 8 顆 LPU。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 171/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0171

Claim: 2 顆 FPGA 各自都有 8× PCIe Gen 5 連到 CPU。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 172/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0172

Claim: LPU 必須經過 FPGA 才能和 CPU 介接，因為 LPU 沒有 PCIe PHY 可以直接連線。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Inter-node/Intra-rack

Each LPU connects to one LPU from each of the 15 other nodes in the server. Each of these inter-node links is 2x100G so there are 15x2x100G inter-node links coming out of each LPU. These inter-node links are via a copper cable backplane. In addition, each FPGA also connects to an FPGA in every other node at either 25G or 50G per link for 15x25G/50G. This also goes through the backplane.

每顆 LPU 都會連到 server 中其他 15 個 node 各一顆 LPU。每條 inter-node link 為 2×100G，因此每顆 LPU 對外共有 15×2×100G inter-node link，並透過 copper cable backplane 傳輸。另外，每顆 FPGA 也會連到其他每個 node 的一顆 FPGA，每條 link 為 25G 或 50G，合計 15×25G／50G，同樣經由 backplane。

Atomic Claim 173/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0173

Claim: 每顆 LPU 都會連到 server 其餘 15 個 node 中各一顆 LPU。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 174/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0174

Claim: 每條 inter-node link 為 2×100G，因此每顆 LPU 對外有 15×2×100G 的 inter-node links。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 175/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0175

Claim: 這些 inter-node links 透過 copper cable backplane 傳輸。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 176/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0176

Claim: 此外，每顆 FPGA 都會連到其他每個 node 中的一顆 FPGA，每條 link 為 25G 或 50G，合計為 15×25G／50G。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 177/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0177

Claim: 這些連線同樣會經過 backplane。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

This means that each node has 16 x 15 x 2 lanes for inter-node C2C and 2 x 15 lanes for inter-node FPGA which is a total of 510 lanes or 1020 differential pairs (for Rx and Tx). Therefore, the backplane is 16 x 1020/2 = 8,160 differential pairs – we divide by 2 as each device Tx channel is a corresponding device’s Rx channel.

每顆 LPU 還會各自連到 server 其餘 15 個 node 中的一顆 LPU。每條 inter-node link 是 2×100G，因此每顆 LPU 有 15×2×100G inter-node link，走 copper cable backplane。除此之外，每顆 FPGA 也會連到其餘每個 node 的一顆 FPGA，每 link 可能是 25G 或 50G，因此總共 15×25G/50G，同樣走 backplane。換算下來，每 node 有 16×15×2 lanes 的 inter-node C2C，再加 2×15 lanes 的 inter-node FPGA，合計 510 lanes，若 Rx、Tx 分開就是 1,020 differential pairs。整個 backplane 因此有 16×1,020÷2 = 8,160 differential pairs；除以 2 是因為一個 device 的 Tx channel 對應另一個 device 的 Rx channel。

Atomic Claim 178/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0178

Claim: 因此，每個 node 都有 16×15×2 lanes 用於 inter-node C2C。
Frame: ATTRIBUTE · Mode: INFERRED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 179/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0179

Claim: inter-node FPGA 則有 2×15 lanes，使總數達到 510 lanes，也就是 1020 differential pairs（Rx 與 Tx）。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 180/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0180

Claim: 因此 backplane 共有 16×1020÷2＝8,160 differential pairs；之所以除以 2，是因為每個 device 的 Tx channel 對應另一個 device 的 Rx channel。
Frame: ATTRIBUTE · Mode: INFERRED · Mapping: COMPLETE
開啟逐條審核

Inter-rack

Source: SemiAnalysis Networking Model ↗

Lastly, there is the inter-rack C2C. Each LPU has 4x100G lanes that go to the OSFP cages to connect LPUs across 4 racks. There are various configurations that can be used for this inter-rack scale up. One option is 4x100G from each LPU going to one OSFP cage, each OSFP escaping 800G of C2C from 2 LPUs. However, for greater fan out the preferred configuration seems to be each 100G lane from the LPU going to 4 individual cages, with each cage escaping 800G of C2C from 8 LPUs.

最後是 inter-rack C2C。每顆 LPU 有 4×100G lane 連到 OSFP cage，用來跨 4 個 rack 連接 LPU。Inter-rack scale-up 可以有多種配置：一種做法是每顆 LPU 的 4×100G 全部接到同一個 OSFP cage，每個 cage 輸出來自兩顆 LPU、合計 800G 的 C2C；若要提高 fan-out，較偏好的配置是讓每顆 LPU 的四條 100G lane 分別進入四個 cage，每個 cage 輸出來自八顆 LPU、合計 800G 的 C2C。

Atomic Claim 181/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0181

Claim: 最後還有 inter-rack C2C。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 182/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0182

Claim: 每顆 LPU 有 4×100G lanes 連到 OSFP cage，用來跨 4 個 rack 連接 LPU。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 183/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0183

Claim: 這種 inter-rack scale-up 可以採用多種不同 configuration。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 184/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0184

Claim: 其中一種 configuration，是每顆 LPU 的 4×100G 連到一個 OSFP cage；每個 OSFP 會從 2 顆 LPU 對外導出 800G 的 C2C。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 185/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0185

Claim: 若要更大的 fan-out，偏好的 configuration 看起來是讓 LPU 的每條 100G lane 分別連到 4 個 cage；每個 cage 會從 8 顆 LPU 對外導出 800G 的 C2C。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

In terms of how the racks are networked together it appears to be a daisy chain configuration, with each Node0 connected to 2 other Node 0. This can all be achieved within the reach of 100G AECs, though optics can be used if necessary.

最後是 inter-rack C2C。每顆 LPU 有 4×100G lane 走到 OSFP cage，用來跨最多 4 個 rack 連接 LPU。Inter-rack scale-up 有多種 configuration。一種做法是每顆 LPU 的 4×100G 全部進同一個 OSFP cage，每個 OSFP 匯出來自 2 顆 LPU、合計 800G C2C。但若追求更大 fan-out，目前比較偏好的配置，是把每顆 LPU 的四條 100G lane 分別送到四個獨立 cage，每個 cage 匯出 8 顆 LPU 各一條 100G、合計 800G C2C。Rack-to-rack topology 看起來會採 daisy chain，例如每個 Node0 連到另外兩個 Node0。這些距離基本上都能用 100G AEC 覆蓋，必要時也可用 optics。

Atomic Claim 186/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0186

Claim: 關於 LPX：機櫃之間看起來採用 daisy-chain 方式串接 together。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 187/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0187

Claim: 每個 Node0 會連接到另外 2 個 Node0。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 188/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0188

Claim: 上述連線都可以在 100G AEC 的傳輸距離內完成；必要時也可以使用 optics。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟逐條審核

Nvidia’s CPO Roadmap

NVIDIA revealed its CPO Roadmap at the GTC Keynote 2026, with Jensen following up with additional commentary in the Financial Analyst Q+A meeting held the following day. Though many had their hopes up for CPO to be used for scale-up within the rack for Rubin Ultra Kyber, Nvidia’s focus was instead on using CPO to enable larger world size compute systems.

NVIDIA 在 GTC 2026 Keynote 公布 CPO roadmap，Jensen 隔天在 Financial Analyst Q+A 又補充更多說明。很多人原本期待 Rubin Ultra Kyber rack 內的 scale-up 就會使用 CPO，但 Nvidia 的重點其實是用 CPO 建立更大的 world-size compute system。

Atomic Claim 189/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0189

Claim: NVIDIA 在 2026 年 GTC Keynote 公布 CPO roadmap，Jensen 並在隔天的 Financial Analyst Q+A meeting 補充評論。
Frame: RELATION · Mode: ATTRIBUTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 190/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0190

Claim: 雖然市場原本期待 CPO 用於 Rubin Ultra Kyber rack 內的 scale-up，Nvidia 的重點反而是用 CPO 支援更大的 world-size compute system。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

In the Rubin Generation, Nvidia will offer the Rubin GPU in an Oberon NVL72 form factor with an all-copper scale-up network. For Rubin Ultra, as we expected, there will only be a copper scale-up option for Rubin Ultra in the Oberon and Kyber Rack form factor. Rubin Ultra will also be offered in a larger world size system that connects 8 Oberon Racks of 72 Rubin Ultra GPUs to form what will be referred to as NVL576.

在 Rubin generation，Nvidia 會以 Oberon NVL72 form factor 提供 Rubin GPU，scale-up network 全部採 copper。到了 Rubin Ultra，如我們預期，Oberon 與 Kyber rack form factor 也只會提供 copper scale-up option。Rubin Ultra 另會有更大的 world-size system，把 8 個 Oberon rack、每 rack 72 顆 Rubin Ultra GPU 連接起來，形成所謂 NVL576。

Atomic Claim 191/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0191

Claim: 在 Rubin generation，Nvidia 將以 Oberon NVL72 form factor 提供 Rubin GPU，並採全 copper 的 scale-up network。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 192/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0192

Claim: 對 Rubin Ultra 而言，在 Oberon 與 Kyber Rack form factor 中只會提供 copper scale-up 選項。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 193/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0193

Claim: Rubin Ultra 也會提供較大 world-size 的系統：將 8 個 Oberon Rack、每櫃 72 顆 Rubin Ultra GPU 連在一起，形成 NVL576。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

CPO scale-up will be used to build the larger world size, connecting between the racks in a two-tier all to all network, though scale-up inside the racks will remain copper-based.

Rubin 世代，Nvidia 會以 Oberon NVL72 form factor 提供 Rubin GPU，rack 內 scale-up 全部使用 copper。Rubin Ultra 方面，如我們原先預期，Oberon、Kyber rack form factor 的 rack 內 scale-up 也只有 copper option。不過 Rubin Ultra 還會提供更大的 world-size system：把 8 個 Oberon rack、每 rack 72 顆 Rubin Ultra GPU 串起來，形成 NVL576。這種較大 world size 會使用 CPO scale-up，在 rack 之間用 two-tier all-to-all network 互連；但 rack 內 scale-up 仍維持 copper。

Atomic Claim 194/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0194

Claim: 較大的 world-size 系統會用 CPO scale-up 做 rack 間連接，採 two-tier all-to-all network；rack 內的 scale-up 則仍維持 copper。
Frame: COMPARISON · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

When we reach the Feynman Generation, CPO usage will expand via another large world size rack, the NVL1152 which is formed by combining 8 Kyber racks.

進入 Feynman generation 後，CPO 的使用會透過另一種大型 world-size rack 進一步擴大，也就是由 8 個 Kyber rack 組成的 NVL1152。

Atomic Claim 195/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0195

Claim: 進入 Feynman generation 後，CPO 的使用會擴展到另一個大型 world-size rack：NVL1152，由 8 個 Kyber rack 組成。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

While the Nvidia Technical Blog that outlines the rack configuration roadmap states that “NVIDIA Kyber will scale up into a massive all-to-all NVL1152 supercomputer using similar direct optical interconnects for rack-to-rack scale-up”, Jensen Huang in a Financial Analyst Q+A session did say that NVL1152 in Feynman would be “all CPO”. There is some disagreement on whether copper will still be used for scale-up within the rack or whether CPO will replace copper.

到了 Feynman 世代，CPO 使用範圍會進一步擴張，推出另一套更大 world-size system——NVL1152，由 8 個 Kyber rack 組成。Nvidia Technical Blog ↗ 的 rack roadmap 寫著：『NVIDIA Kyber will scale up into a massive all-to-all NVL1152 supercomputer using similar direct optical interconnects for rack-to-rack scale-up』；但 Jensen 在 Financial Analyst Q+A 又說 Feynman NVL1152 會是『all CPO』。目前仍存在分歧：rack 內 scale-up 是否繼續使用 copper，還是 CPO 會連 rack 內也一併取代。

Atomic Claim 196/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0196

Claim: Nvidia Technical Blog 表示 Kyber 會透過 direct optical interconnect 做 rack-to-rack scale-up，擴展成 all-to-all NVL1152；但 Jensen Huang 在 Financial Analyst Q+A 表示 Feynman 的 NVL1152 會是「all CPO」。
Frame: CLAIM_ONLY · Mode: ATTRIBUTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 197/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0197

Claim: 目前對 rack 內 scale-up 到底會繼續使用 copper，還是由 CPO 取代 copper，存在不同說法。
Frame: COMPARISON · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Nvidia’s approach has been to use copper where they can, and optics where they must. The architecture of NVL1152 in the Feynman generation will follow the same principle. It is clear that the NVL1152 will adopt CPO to connect between racks, but from GPUs to NVLink Switches is currently copper POR. Nvidia is unable to achieve another doubling of electrical lane speed from 224Gbit/s bi-di to 448Gbit/s uni-di means bandwidth isn’t that amazing.

Nvidia 一貫策略是『能用 copper 就用 copper，非用 optics 不可才用 optics』，Feynman NVL1152 也會遵循同樣原則。Rack-to-rack 已確定會導入 CPO；但目前 GPU 到 NVLink Switch 的 plan-of-record 仍是 copper。Nvidia 無法再把 electrical lane speed 從 224Gbit/s bi-di 直接翻倍到 448Gbit/s uni-di，因此 bandwidth 提升沒有想像中驚人。

Atomic Claim 198/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0198

Claim: Nvidia 一貫的原則，是能使用 copper 的地方就優先使用銅互連。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 199/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0199

Claim: ……只有在不得不用的地方才使用 optics。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 200/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0200

Claim: Feynman generation 的 NVL1152 architecture 也會遵循相同原則。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 201/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0201

Claim: 可以確定 NVL1152 將採用 CPO 進行機櫃間連接。
Frame: RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 202/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0202

Claim: 目前從 GPU 到 NVLink Switch 的 POR 仍採 copper。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 203/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0203

Claim: Nvidia 無法再把 electrical lane speed 從 224Gbit/s bi-di 翻倍到 448Gbit/s uni-di，因此可提供的 bandwidth 並沒有大幅提升。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

While 448G high speed SerDes have big challenges for shoreline, reach, and power versus using a die-to-die connection to an optical engine, the manufacturing challenges, cost, and reliability for Feynman necessitate using copper to the Switch.

448G high-speed SerDes 相較 die-to-die connection + optical engine，在 shoreline、reach、power 上都有很大挑戰；但 Feynman 若考量 manufacturing difficulty、cost、reliability，仍迫使 Nvidia 在 GPU-to-switch 這段優先使用 copper。

Atomic Claim 204/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0204

Claim: 448G high-speed SerDes 相較 die-to-die 連到 optical engine，在 shoreline、reach 與 power 上有很大挑戰；但考量 Feynman 的製造難度、成本與可靠性，仍需要使用 copper 連到 switch。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

With that said, the NVL1152 SKU is years out – and the roadmap is highly likely to shift. For now, our base case stands at copper being used within each rack and CPO between the racks, but this could easily change.

當然，NVL1152 SKU 還要幾年才會真正出現，roadmap 很可能繼續變。目前我們的 base case 還是 rack 內 copper、rack 間 CPO，但這完全可能改變。

Atomic Claim 205/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0205

Claim: NVL1152 SKU 距離實際推出仍有數年。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 206/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0206

Claim: 因此該 roadmap 很可能還會改變。
Frame: CLAIM_ONLY · Mode: EXPECTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 207/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0207

Claim: 目前 SemiAnalysis 的 base case 是：每個機櫃內使用 copper，機櫃之間使用 CPO。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 208/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0208

Claim: 不過，這個 base case 也很容易發生變化。
Frame: CLAIM_ONLY · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟逐條審核

For now – our best estimate of Nvidia’s CPO roadmap is as follows:

目前我們對 Nvidia CPO roadmap 的最佳估計如下：

Atomic Claim 209/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0209

Claim: 目前 SemiAnalysis 對 Nvidia CPO roadmap 的最佳估計如下。
Frame: ATTRIBUTE · Mode: ESTIMATED · Mapping: PARTIAL
開啟逐條審核

Rubin:

NVL72 – Oberon all copper scale up

NVL72：Oberon，全 copper scale-up。

Atomic Claim 210/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0210

Claim: NVL72 採 Oberon，scale-up 全部使用 copper。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Rubin Ultra:

NVL72 – Oberon all copper scale up

NVL72：Oberon，全 copper scale-up。

Atomic Claim 211/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0211

Claim: NVL72 採 Oberon，scale-up 全部使用 copper。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

NVL144 – Kyber rack all copper scale up

NVL144：Kyber rack，全 copper scale-up。

Atomic Claim 212/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0212

Claim: NVL144 採 Kyber rack，scale-up 全部使用 copper。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

NVL288 – Kyber rack all copper scale up with copper connecting 2 racks together

NVL288：Kyber rack，全 copper scale-up，並用 copper 把兩個 rack 互連。

Atomic Claim 213/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0213

Claim: NVL288 採 Kyber rack，scale-up 全部使用 copper，並以 copper 將兩個機櫃連接 together。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

NVL576 – 8x Oberon Racks copper scale up within rack and CPO on switch between racks in a two tier all to all topology. This would be low volume for test purposes

NVL576：由 8 個 Oberon rack 組成；rack 內使用 copper scale-up，rack 間 switch 使用 CPO，以 two-tier all-to-all topology 互連。這個版本預計只會低量用於 testing。

Atomic Claim 214/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0214

Claim: NVL576 由 8 個 Oberon Rack 組成；rack 內 scale-up 使用 copper，rack 間則在 switch 上使用 CPO，形成 two-tier all-to-all topology。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 215/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0215

Claim: 關於 CPO：這種配置預期只會以低量方式用於測試。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Feynman:

NVL72 – Oberon Rack – All Copper

NVL72：Oberon Rack，全 copper。

Atomic Claim 216/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0216

Claim: Feynman NVL72 採 Oberon rack，全部使用 Copper。
Frame: RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

NVL144 – Kyber Rack – All Copper

NVL144：Kyber Rack，全 copper。

Atomic Claim 217/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0217

Claim: Feynman NVL144 採 Kyber Rack，全部使用 Copper。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

NVL1152 – 8xKyber Rack – Copper within rack and CPO on the switch between racks

NVL1152：8 個 Kyber Rack；rack 內使用 copper，rack 間 switch 使用 CPO。

Atomic Claim 218/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0218

Claim: Feynman NVL1152 由 8 個 Kyber racks 組成，機櫃內採 Copper，機櫃間則在 switch 上使用 CPO。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Source: SemiAnalysis, Nvidia

Oberon and Kyber Updates, Larger World Sizes Introduced, More Networking Updates

Nvidia provided a long-awaited update on its Kyber rack form factor, the latest addition to the lineup after Oberon having first been previewed as a prototype at GTC 2025. As a prototype, the rack architecture has continued to evolve, and we notice some changes. First, each compute blade has densified, with 4x Rubin Ultra GPU and 2x Vera each. There are a total of 2 canisters of 18 compute blades which amounts to 36 compute blades total for 144 GPUs in a rack.

Nvidia 終於更新了 Kyber rack form factor。Kyber 是 Oberon 之後的最新 rack 設計，最早在 GTC 2025 以 prototype 形式展示；隨著 prototype 持續演進，rack architecture 也出現一些變化。首先，每個 compute blade 的密度提高到 4 顆 Rubin Ultra GPU + 2 顆 Vera。整個 rack 有兩個 canister，每個 canister 18 個 compute blade，合計 36 個 compute blade、144 顆 GPU。

Atomic Claim 219/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0219

Claim: Nvidia 更新了 Kyber rack form factor；這是 Oberon 在 GTC 2025 首次以 prototype 預覽之後，產品線中最新加入的 form factor。
Frame: RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 220/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0220

Claim: 由於仍處於 prototype 階段，這個 rack architecture 持續演進，並出現了一些變化。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 221/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0221

Claim: 首先，每個 compute blade 的密度提高，配置 4 顆 Rubin Ultra GPU 與 2 顆 Vera。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 222/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0222

Claim: 每個 rack 有 2 個 canister，每個 canister 含 18 個 compute blade，因此共 36 個 compute blade、合計 144 顆 GPU。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

The initial Kyber design featured 2 GPUs and 2 Vera CPUs in one compute blade, with a total of 4 canisters of 18 compute blades each.

Nvidia 終於更新了大家等很久的 Kyber rack form factor。Kyber 是 Oberon 之後最新加入 lineup 的 rack architecture，最早在 GTC 2025 以 prototype 形式亮相。既然是 prototype，architecture 一直持續演進，現在已看到幾個改變。第一，compute blade density 提高：每 blade 改成 4×Rubin Ultra GPU + 2×Vera。整 rack 有兩個 canister、每個 18 個 compute blade，共 36 個 blade、144 顆 GPU。最初 Kyber design 則是每 compute blade 2 顆 GPU + 2 顆 Vera CPU，總共四個 canister、每個 18 blade。

Atomic Claim 223/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0223

Claim: 最初的 Kyber design，在每個 compute blade 中配置 2 顆 GPU 與 2 顆 Vera CPU，整櫃則有 4 個 canister、每個 canister 18 個 compute blade。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

The details below are based on the Rubin Kyber prototypes, but Rubin Ultra will be redone.

下面細節基於 Rubin Kyber prototype，但 Rubin Ultra 版本還會重新設計。

Atomic Claim 224/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0224

Claim: 以下細節是基於 Rubin Kyber prototypes。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 225/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0225

Claim: Rubin Ultra 的設計將重新調整。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Source: SemiAnalysis

Each switch blade is also double in height vs the GTC 2025 prototype, with 6 NVLink 7 switches per switch blade, and 12 switch blades per rack, amounting to a total of 72 NVLink 7 switches per Kyber rack. The GPUs are connected all-to-all to the switch blades via 2 PCB midplanes or 1 midplane per canister.

每個 switch blade 相較 GTC 2025 prototype 高度也加倍，每 blade 放 6 顆 NVLink 7 switch，整 rack 有 12 個 switch blade，因此 Kyber rack 總計 72 顆 NVLink 7 switch。GPU 透過兩塊 PCB midplane 和 switch blade 做 all-to-all connection，也就是每個 canister 配一塊 midplane。

Atomic Claim 226/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0226

Claim: 每個 switch blade 的高度也比 GTC 2025 prototype 增加一倍；每個 switch blade 配置 6 顆 NVLink 7 switch，每個 rack 有 12 個 switch blade，因此每個 Kyber rack 總共有 72 顆 NVLink 7 switch。
Frame: COMPARISON · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 227/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0227

Claim: GPU 會以 all-to-all 方式連到 switch blade，透過 2 個 PCB midplane，或等價地每個 canister 1 個 midplane。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Kyber midplane PCB (GPU side). Source: Nvidia, SemiAnalysis

Kyber midplane PCB（GPU side）。Source: Nvidia, SemiAnalysis

Atomic Claim 228/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0228

Claim: Kyber midplane PCB（原文標記為 §3§ side）。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 229/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0229

Claim: 來源：Nvidia、SemiAnalysis。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

For Rubin Ultra NVL144 Kyber, there will be no CPO used for scale up, despite rumors from other analysts suggesting scale-up CPO introduction for Kyber. However, optics for NVLink are coming and will be progressively phased in. Scale-up CPO will first be used for the Rubin Ultra NVL576 system to connect between 8 Oberon form factor racks, forming a two-layer all-to-all network.

Rubin Ultra NVL144 Kyber 的 scale-up 不會使用 CPO，與其他 analyst 傳聞 Kyber 將導入 scale-up CPO 的說法不同。不過 NVLink optics 確實會到來，並逐步導入。Scale-up CPO 會先用在 Rubin Ultra NVL576 system，用來連接 8 個 Oberon form-factor rack，形成 two-layer all-to-all network。

Atomic Claim 230/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0230

Claim: 對 Rubin Ultra NVL144 Kyber 而言，scale-up 不會使用 CPO；這與其他分析師傳出的 Kyber 將導入 scale-up CPO 的說法不同。
Frame: NARY_RELATION · Mode: RUMORED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 231/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0231

Claim: 不過，NVLink 的光學連線即將導入。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 232/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0232

Claim: ……之後會逐步導入。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 233/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0233

Claim: scale-up CPO 會先用在 Rubin Ultra NVL576，連接 8 個 Oberon form-factor rack，形成 two-layer all-to-all network。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

A copper backplane will still be used for scale-up networking within the racks however. This is still for low volume / testing purposes.

Rubin Ultra NVL144 Kyber 不會在 rack 內 scale-up 使用 CPO，這點我們已多次告訴 client ↗，儘管其他 analyst 一直有傳言說 Kyber 會導入 scale-up CPO。不過 NVLink optics 的確會來，而且會逐步導入。Scale-up CPO 首先會用在 Rubin Ultra NVL576：把 8 個 Oberon form-factor rack 互連，形成 two-layer all-to-all network；rack 內 scale-up 仍使用 copper backplane。這個版本目前仍以低量／testing 為主。

Atomic Claim 234/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0234

Claim: 不過 rack 內的 scale-up networking 仍會使用 copper backplane。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 235/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0235

Claim: 這仍然只會用於低量／測試用途。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Moving back to the Kyber Rack, each Rubin Ultra logical GPU offers 14.4Tbit/s uni-di of scale-up bandwidth, using an 80DP connector (72 DPs used x 200Gbit/s bi-di channel = 14.4Tbit/s) per GPU for connectivity to the midplane board. Connecting all 144 GPUs in an all-to-all network will require 72 NVLink 7.0 Switch Chips running at 28.8Tbit/s uni-di of aggregate bandwidth each.

回到 Kyber Rack，每顆 Rubin Ultra logical GPU 提供 14.4Tbit/s uni-di scale-up bandwidth，透過每 GPU 一個 80DP connector 連到 midplane board；其中使用 72 DP × 200Gbit/s bi-di channel = 14.4Tbit/s。若要把 144 顆 GPU 全部連成 all-to-all network，需要 72 顆 NVLink 7.0 Switch Chip，每顆 aggregate bandwidth 28.8Tbit/s uni-di。

Atomic Claim 236/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0236

Claim: 回到 Kyber Rack，每顆 Rubin Ultra logical GPU 可提供 14.4Tbit/s uni-di scale-up bandwidth；每顆 GPU 透過 80DP connector（使用 72 DPs × 200Gbit/s bi-di channel＝原文 §4§）連到 midplane board。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 237/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0237

Claim: 要把 144 顆 GPU 以 all-to-all network 互連，需要 72 顆 NVLink 7.0 switch chip，每顆提供 28.8Tbit/s uni-di aggregate bandwidth。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Source: SemiAnalysis

In the Kyber Switch Blade picture below, we can see that there are 2 separate PCBs carrying 3 Switches each. The switch blade should have 6 152DP connectors, 3 connectors serving each midplane board. The picture is a prototype blade using less dense connectors, which is why there are 12 connectors instead of the 6 that we expect in the production version.

從下方 Kyber Switch Blade 圖可以看到，實際上有兩塊獨立 PCB，每塊各放 3 顆 Switch。Production switch blade 應會有 6 個 152DP connector，每三個 connector 對應一塊 midplane board。照片中的 prototype blade 使用較低 density connector，所以才會看到 12 個 connector，而不是我們預期 production version 的 6 個。

Atomic Claim 238/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0238

Claim: 從 Kyber switch-blade 圖片可看到兩片獨立 PCB，每片承載 3 顆 switch。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 239/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0239

Claim: switch blade 預計會有 6 個 152DP connector，其中每個 midplane board 由 3 個 connector 服務。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 240/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0240

Claim: 圖片中的 prototype blade 使用較低密度 connector，因此有 12 個 connector；production version 預期則會是 6 個。
Frame: CLAIM_ONLY · Mode: EXPECTED · Mapping: PARTIAL
開啟逐條審核

Source: Nvidia, SemiAnalysis

Each 28.8T NVLink Switch has 144 lanes of 200G (simultaneous bi-directional) which means each Switch has 24 lanes of 200G going to each connector. Copper flyover cables are used to connect each switch to the midplane, as the distances involved are too long for PCB traces. This is also why the switches are further away from the midplane, to provide space for the routing of the flyover cables.

每顆 28.8T NVLink Switch 有 144 lanes × 200G simultaneous bi-directional，因此每顆 Switch 會分 24 條 200G lane 到每個 connector。Switch 到 midplane 使用 copper flyover cable，因為距離太長，PCB trace 無法承受。這也是為什麼 switch 會刻意離 midplane 遠一點，以保留 flyover cable routing 空間。

Atomic Claim 241/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0241

Claim: 每顆 28.8T NVLink Switch 有 144 lanes 的 200G simultaneous bi-directional link，因此每顆 switch 會有 24 lanes 的 200G 連到每個 connector。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 242/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0242

Claim: 使用 Copper flyover cable 將每顆 switch 連到 midplane，因為距離太長，不適合使用 PCB trace。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 243/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0243

Claim: 這也是 switch 與 midplane 距離較遠的原因，以便留下空間配置 flyover cable。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Each NVLink Switch Chip connects via flyover cables to the connector (144 DPs used x 200 Gbit/s bi-di channel = 28.8Tbit/s) connectors at the edge of the switch blade, and these connectors plug into the midplane board. Nvidia is looking into using co-packaged Copper to reduce loss further, in case NPC doesn’t work. As far as we know Nvidia is telling supply chain to go for fully co-packaged copper.

每顆 NVLink Switch Chip 透過 flyover cable 連到 switch blade 邊緣 connector；完整使用 144 DP × 200Gbit/s bi-di channel = 28.8Tbit/s。這些 connector 再插入 midplane board。Nvidia 也在研究 co-packaged copper，以便如果 NPC 不可行時進一步降低 loss；就我們所知，Nvidia 現在給 supply chain 的方向是全面朝 co-packaged copper。

Atomic Claim 244/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0244

Claim: 每顆 NVLink Switch chip 透過 flyover cable 連到 switch blade 邊緣的 connector（使用 144 DPs × 200Gbit/s bi-di＝28.8Tbit/s），再由 connector 插入 midplane board。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 245/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0245

Claim: Nvidia 正在評估使用 co-packaged Copper 進一步降低 loss，以防 NPC 無法運作。
Frame: RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 246/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0246

Claim: 依 SemiAnalysis 目前掌握，Nvidia 正要求 supply chain 朝 fully co-packaged copper 的方向準備。
Frame: RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Rubin Ultra NVL288

Though not officially discussed by Nvidia at GTC 2026, an NVL288 concept has been explored within the supply chain. This would entail two NVL144 Kyber racks placed adjacent to each other, with a rack-to-rack copper backplane used to connect the two racks. One possibility is that all 288 GPUs are connected all to all, but this would require higher radix switches than the current NVLink 7 switches which only offer a maximum radix of 144 ports of 200G.

Nvidia 雖沒有在 GTC 2026 正式談 NVL288，但 supply chain 內部確實探索過這個 concept：把兩個 NVL144 Kyber rack 並排，以 rack-to-rack copper backplane 互連。其中一種可能是讓 288 顆 GPU 全部 all-to-all，但這需要比現有 NVLink 7 更高 radix 的 switch，因為目前 NVLink 7 maximum radix 只有 144 個 200G port。

Atomic Claim 247/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0247

Claim: 雖然 Nvidia 並未在 GTC 2026 正式討論，但 supply chain 內部已探索過 NVL288 concept。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 248/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0248

Claim: 此概念會將兩個 NVL144 Kyber racks 並排放置。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 249/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0249

Claim: 會使用 rack-to-rack copper backplane 連接這兩個 rack。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 250/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0250

Claim: 其中一種可能方案，是讓全部 288 顆 GPUs 彼此 all-to-all 連接。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 251/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0251

Claim: 這種方案需要比目前 NVLink 7 更高 radix 的 switch，因為現有 NVLink 7 最大只提供 144 個 200G port。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

If Rubin Ultra NVL288 is deployed, each Rubin Ultra GPU will have a scale-up bandwidth of 14.4Tbit/s uni-di, requiring 144 DPs of cables to connect the NVLink 7 switches. 72 DPs per GPU times 288 GPUs means a total of 20,736 additional DPs required to connect this larger world size domain. This entails a lot of cables, so it is an upper bound of how much cable content could be used.

如果 Rubin Ultra NVL288 真正部署，每顆 Rubin Ultra GPU 仍有 14.4Tbit/s uni-di scale-up bandwidth，需要 144 DP cable 連 NVLink 7 switch。72 DP/GPU × 288 GPU，總計需要額外 20,736 DP 才能支援這個更大 world-size domain。Cable 數量非常驚人，因此這可以視為 copper content 的上限情境。

Atomic Claim 252/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0252

Claim: 若部署 Rubin Ultra NVL288，每顆 Rubin Ultra GPU 會有 14.4Tbit/s uni-di scale-up bandwidth，並需要 144 DPs cable 連接 NVLink 7 switch。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 253/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0253

Claim: 每顆 GPU 72 DPs × 288 顆 GPU，代表較大 world-size domain 需要額外 20,736 DPs。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 254/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0254

Claim: 這會需要大量 cable，因此可視為 cable content 使用量的上限估計。
Frame: CLAIM_ONLY · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟逐條審核

The radix of the 28.8T NVLink Switch limits the number of GPUs that each switch can connect while still providing for cross-rack connectivity. Either a higher radix switch will have to be used - or there will have to be a degree of oversubscription in this architecture while potentially adopting a dragonfly-like network topology. This would also require fewer DPs worth of copper cables.

28.8T NVLink Switch 的 radix 會限制每顆 switch 能連多少 GPU，同時還要保留 cross-rack connectivity。解法不是換更高 radix switch，就是 architecture 必須接受一定程度 oversubscription，甚至改採類 dragonfly network topology；後者也會減少需要的 copper cable DP 數量。

Atomic Claim 255/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0255

Claim: 28.8T NVLink Switch 的 radix 會限制每顆 switch 能連接的 GPU 數量，同時還必須保留 cross-rack connectivity。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 256/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0256

Claim: 因此要嘛使用更高 radix 的 switch，要嘛讓 architecture 出現一定程度的 oversubscription，並可能採用類似 dragonfly 的 network topology。
Frame: CLAIM_ONLY · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 257/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0257

Claim: 這樣也會減少所需的 DP 數量與 copper cable content。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Source: SemiAnalysis

All current evidence in the supply chain points to NVSwitch 7 being the same bandwidth as NVSwitch 6, but that seems a bit illogical to be frank. Our belief is that NVSwitch 7 is actually 2x the bandwidth and radix of NVSwitch 6, so all-to-all can be done, and architecturally that makes the most sense from a systems perspective.

目前 supply chain 所有 evidence 都指向 NVSwitch 7 bandwidth 和 NVSwitch 6 相同，但坦白說這有點不合理。我們更傾向認為 NVSwitch 7 實際上會有 NVSwitch 6 的 2x bandwidth、2x radix，這樣才有辦法做 all-to-all；從 system architecture 的角度看也最合理。

Atomic Claim 258/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0258

Claim: 目前所有供應鏈證據都指向 NVSwitch 7 的頻寬與 NVSwitch 6 相同。
Frame: COMPARISON · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 259/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0259

Claim: 坦白說，這樣的結果看起來有些不合邏輯。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 260/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0260

Claim: SemiAnalysis 的判斷是，NVSwitch 7 實際上可能具有 2 倍的 bandwidth……
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 261/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0261

Claim: ……相較 NVSwitch 6 的 radix，如此才能實現 all-to-all；從 system architecture 角度看，這也最合理。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Rubin Ultra NVL576

To push the scale up world size beyond 144 GPUs and across multiple racks, optics are needed as we are approaching the maximum compute density that is within the reach of copper. Rubin Ultra NVL576 is now on the roadmap with 8 racks of lower density Oberon.

要把 scale-up world size 從 144 GPU 再往上推、跨越多個 rack，就需要 optics，因為 compute density 已接近 copper reach 的極限。Rubin Ultra NVL576 現在已在 roadmap 上，由 8 個較低 density 的 Oberon rack 組成。

Atomic Claim 262/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0262

Claim: 若要把 scale-up world size 推進到超過 144 顆 GPU……
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 263/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0263

Claim: ……跨越多個 rack 時就需要 optics，因為 compute density 已接近 copper 傳輸距離所能支援的上限。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 264/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0264

Claim: Rubin Ultra NVL576 現已列入 roadmap，配置 8 個密度較低的 Oberon rack。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Source: SemiAnalysis

Optics will be required for the inter-rack connections, though strictly speaking it isn’t confirmed whether this will be with pluggable optics or with CPO, though CPO seems much more likely. The current Blackwell NVL576 prototype “Polyphe” uses pluggable optics.

Rack 間 connection 一定需要 optics；嚴格來說，目前還沒正式確認會用 pluggable optics 還是 CPO，但 CPO 看起來明顯更可能。目前 Blackwell NVL576 prototype『Polyphe』使用的是 pluggable optics。

Atomic Claim 265/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0265

Claim: inter-rack connection 需要 optics，但目前尚未確認會使用 pluggable optics 還是 CPO；其中 CPO 看起來可能性高得多。
Frame: COMPARISON · Mode: EXPECTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 266/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0266

Claim: 目前 Blackwell NVL576 prototype「Polyphe」使用 pluggable optics。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

We have shown a concept of NVL576 for GB200 previously with pluggable optics to interconnect the second layer of NVLink switches. The use of pluggables contributed to an enormous increase in BOM cost that made the system untenable from a TCO perspective for a switched all-to-all. However, it is plausible that Rubin Ultra NVL576 will be rolled out in test volumes before Feynman NVL1152, where we will see actual volume ramp of scale-up CPO.

我們之前已展示過 GB200 NVL576 concept ↗，以 pluggable optics 連接第二層 NVLink switch。Pluggable 帶來非常巨大的 BoM cost increase，讓 switched all-to-all system 從 TCO 角度幾乎不可行。不過 Rubin Ultra NVL576 很可能仍會先用 test volume 推出，替之後 Feynman NVL1152 真正大量 ramp scale-up CPO 做準備。

Atomic Claim 267/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0267

Claim: SemiAnalysis 先前曾展示 GB200 的 NVL576 concept，使用 pluggable optics 連接第二層 NVLink switch。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 268/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0268

Claim: 使用 pluggable 會大幅提高 BOM cost，使 switched all-to-all system 從 TCO 角度變得不可行。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 269/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0269

Claim: 合理的情境是：Rubin Ultra NVL576 先以 test volume 推出，之後到 Feynman NVL1152 才會看到 scale-up CPO 真正的 volume ramp。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

The downstream implications of this are exposed in our institutional research, trusted by all major hyperscalers, semiconductor companies, and AI Labs, at sales@semianalysis.com.

這些 downstream implication 都收錄在我們的 institutional research 中，主要 hyperscaler、semiconductor company、AI Lab 都有使用；如需洽詢可聯絡 sales@semianalysis.com。

Atomic Claim 270/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0270

Claim: 這些 downstream implication 收錄在 SemiAnalysis 的 institutional research；原文並宣稱該研究受到主要 hyperscaler、semiconductor company 與 AI Lab 採用。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Feynman

While not much is known about Feynman, the Keynote sneak peek was enough to tell us Feynman will be exciting, with three major technical innovations all being pushed in a single platform: Hybrid bonding/SoIC, A16, CPO, and custom HBM.

雖然現在對 Feynman 還知道不多，但 Keynote 的 sneak peek 已經足以看出它會很精彩：單一 platform 同時推進多項重大 technical innovation，包括 Hybrid bonding／SoIC ↗、A16、CPO ↗、custom HBM ↗。

Atomic Claim 271/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0271

Claim: 雖然目前對 Feynman 所知仍不多，但 keynote 的 sneak peek 足以讓 SemiAnalysis 認為 Feynman 很值得期待。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 272/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0272

Claim: 單一平台同時推進多項主要技術創新：Hybrid bonding／SoIC、A16、CPO，以及 custom HBM。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

While Feynman adopting CPO is on the roadmap, the question is to what extent? Will in-rack interconnectivity be copper based or optical? We will show possible configurations behind the Paywall.

Feynman 導入 CPO 已經在 roadmap 上，真正問題是會導入到什麼程度：rack 內 interconnect 仍使用 copper，還是也會改成 optical？Paywall 後我們會展示幾種可能 configuration。接著先談 Vera ETL256。

Atomic Claim 273/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0273

Claim: Feynman 採用 CPO 已在 roadmap 上，但實際導入範圍仍未確定。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 274/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0274

Claim: 機櫃內 interconnectivity 究竟會使用 copper 還是 optical，仍是未定問題。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 275/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0275

Claim: SemiAnalysis 表示，將在 paywall 後展示可能的配置方案。
Frame: CLAIM_ONLY · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟逐條審核

Vera ETL256

CPU demand is rising as AI workloads require more data handling, preprocessing, and orchestration beyond GPU compute. Reinforcement learning further increases demand, with CPUs running simulations, executing code, and verifying outputs in parallel. As GPUs scale faster than CPUs, larger CPU clusters are needed to keep them fully utilized, making CPUs a growing bottleneck.

AI workload 不只需要 GPU compute，還需要更多 data handling、preprocessing、orchestration，因此 CPU demand 正快速上升。Reinforcement learning 又會進一步放大 CPU 需求，因為 CPU 必須平行執行 simulation、code execution、output verification。當 GPU scale 速度快於 CPU，為了讓 GPU 保持 fully utilized，就需要更大的 CPU cluster，CPU 也逐漸成為新的 bottleneck。

Atomic Claim 276/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0276

Claim: 隨著 AI workload 在 GPU compute 之外需要更多資料處理、preprocessing 與 orchestration，CPU demand 正在上升。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 277/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0277

Claim: Reinforcement learning 進一步推升 CPU demand，因為 CPU 會平行執行 simulation、code execution 與 output verification。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 278/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0278

Claim: 由於 GPU 的擴張速度快於 CPU，需要更大的 CPU cluster 才能維持 GPU 充分利用，使 CPU 成為日益明顯的 bottleneck。
Frame: COMPARISON · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

The Vera standalone rack addresses this directly, achieving unprecedented density by fitting 256 CPUs into a single rack — a feat that necessitates liquid cooling. The underlying rationale mirrors the NVL rack design philosophy: pack compute tightly enough that copper interconnects can reach everything within the rack, eliminating the need for optical transceivers on the spine. The cost savings from copper more than offset the additional cooling overhead.

Vera standalone rack 就是直接針對這個問題：單一 rack 塞進 256 顆 CPU，density 前所未見，也因此必須 liquid cooling。背後邏輯和 NVL rack design philosophy 一樣——把 compute 壓得夠密，讓 rack 內所有 connection 都還在 copper reach 內，就可以拿掉 spine 上的 optical transceiver。Copper 帶來的 cost saving 足以抵銷額外 cooling overhead。

Atomic Claim 279/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0279

Claim: Vera standalone rack 透過單一 rack 容納 256 顆 CPU 提升密度，而這種密度需要 liquid cooling。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 280/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0280

Claim: 其設計邏輯與 NVL rack 類似：把 compute 壓得足夠緊密，使 copper interconnect 能覆蓋整個 rack，從而不需要 spine 上的 optical transceivers。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 281/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0281

Claim: copper 帶來的成本節省，超過額外 cooling overhead。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Source: SemiAnalysis

Each Vera ETL rack consists of 32 compute trays, 16 above and 16 below, arranged symmetrically around four 1U MGX ETL switch trays (based on Spectrum-6) in the middle. The symmetric split is deliberate: it minimizes cable length variance between compute trays and the spine, keeping all connections within copper reach. From each switch tray, rear-facing ports connect to that copper spine for intra-rack communication, while 32 front-facing OSFP cages provide optical connectivity to the rest of the POD.

每個 Vera ETL rack 有 32 個 compute tray，上下各 16 個，對稱排列在 rack 中央四個 1U MGX ETL switch tray 周圍；這些 switch tray 基於 Spectrum-6。對稱 layout 是刻意設計，用來縮小不同 compute tray 到 spine 的 cable length variance，確保所有 connection 都維持在 copper reach 內。每個 switch tray 後方 port 連到 copper spine，負責 intra-rack communication；前方則有 32 個 OSFP cage，提供到 POD 其他部分的 optical connectivity。

Atomic Claim 282/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0282

Claim: 每個 Vera ETL rack 由 32 個 compute trays 組成，其中 16 個位於上方……
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 283/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0283

Claim: ……另外 16 個位於下方，並以對稱方式配置在中間四個 1U MGX ETL switch trays 周圍。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 284/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0284

Claim: 這種對稱分割是刻意設計：可降低 compute tray 到 spine 的 cable length variance，讓所有連線維持在 copper reach 內。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 285/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0285

Claim: 每個 switch tray 的後方 ports 連到 copper spine，負責機櫃內通訊。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 286/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0286

Claim: 前方共有 32 個 OSFP cages，用來提供連往其餘 POD 的 optical connectivity。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Networking within the rack uses a Spectrum-X multiplane topology, distributing 200 Gb/s lanes across the four switches to achieve full all-to-all connectivity while maintaining a single network tier. With each compute tray housing 8 Vera CPUs, the result is 256 CPUs per rack, all interconnected over Ethernet through a single, flat network.

Rack 內 networking 使用 Spectrum-X multi-plane topology，把 200Gb/s lane 分散到四台 switch，在維持 single network tier 的同時做到 full all-to-all connectivity。每個 compute tray 有 8 顆 Vera CPU，因此整 rack 共有 256 顆 CPU，全部透過單層、flat Ethernet network 互連。

Atomic Claim 287/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0287

Claim: rack 內 networking 採 Spectrum-X multiplane topology，將 200 Gb/s lanes 分散到四個 switch，以單一 network tier 實現完整 all-to-all connectivity。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 288/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0288

Claim: 每個 compute tray 容納 8 顆 Vera CPU，因此每 rack 共 256 顆 CPU，並透過 Ethernet 以單一 flat network 互連。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Source: SemiAnalysis

Source: Nvidia

CMX and STX

We have written extensively on Nvidia’s CMX, or ICMS platform in our last Rubin piece and Memory Model. Nvidia introduced the STX reference storage rack architecture.

我們在上一篇 Rubin 文章與 Memory Model 已經大量討論 Nvidia CMX／ICMS platform。這次 Nvidia 又推出 STX reference storage rack architecture。

Atomic Claim 289/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0289

Claim: SemiAnalysis 表示，先前已在 Rubin 文章與 Memory Model 中大量討論 Nvidia 的 CMX／ICMS platform。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 290/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0290

Claim: Nvidia 推出了 STX reference storage rack architecture。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

CMX

CMX is NVIDIA’s context memory storage platform. CMX addresses a growing bottleneck in modern inference infrastructure: the rapid expansion of KV Cache required to support long-context and agentic workloads.

CMX 是 NVIDIA 的 context-memory storage platform，針對 modern inference infrastructure 越來越大的 bottleneck：為了支援 long-context、agentic workload，所需 KV cache 正快速膨脹。

Atomic Claim 291/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0291

Claim: CMX 是 NVIDIA 的 context memory storage platform。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 292/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0292

Claim: CMX 用來處理現代 inference infrastructure 日益明顯的 bottleneck：支援 long-context 與 agentic workloads 所需的 KV Cache 快速成長。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

KV cache grows linearly with input sequence length and number of users and is the primary tradeoff when it comes to prefill performance (time to first token). At scale, on-device HBM does not have enough capacity. Host DRAM extends beyond HBM capacity with an additional tier of cache, but also hits limits on total amount per node, memory bandwidth, and network bandwidth. Enter NVMe storage for additional KVcache offload.

KV cache 會隨 input sequence length、user 數線性增加，也是 prefill performance（time-to-first-token）的主要 trade-off。Scale 上去後，on-device HBM capacity 不夠；host DRAM 可以在 HBM 外再加一層 cache，但每 node 可放的總量、memory bandwidth、network bandwidth 最後也會碰到限制。再下一層就是用 NVMe storage 做額外 KV cache offload。

Atomic Claim 293/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0293

Claim: KV cache 會隨輸入 sequence length 與 user 數量線性成長，並成為 prefill performance 的主要 tradeoff。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 294/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0294

Claim: 在大規模情境下，device 上的 HBM 容量不足。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 295/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0295

Claim: Host DRAM 可在 HBM 之外增加一層 cache，但同樣會受到每 node 總容量、memory bandwidth 與 network bandwidth 限制。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 296/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0296

Claim: 因此可使用 NVMe storage，進一步 offload KV cache。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

NVIDIA introduced a “new” intermediate storage “tier G3.5” within the inference memory hierarchy at CES in January. Tier G3.5 NVMe sits in between tier G3 DRAM and tier G4 shared storage (also NVMe, or SATA/SAS SSD, or HDD). Previously referred to as ICMS (Inference Context Memory Storage) and now branded as the CMX platform, this is just another re-brand of storage servers attached to compute servers via Bluefield NICs. The only difference from NVMe architectures is the swap from Connect-X NICs to Bluefield NICs.

NVIDIA 在今年 1 月 CES 引入一個『新』的中間 storage tier——G3.5，位於 inference memory hierarchy 的 tier G3 DRAM 與 tier G4 shared storage 之間；tier G4 本身可以是 NVMe、SATA/SAS SSD 或 HDD。這套東西原本叫 ICMS（Inference Context Memory Storage），現在 rebrand 成 CMX platform，本質上仍是 storage server 透過 BlueField NIC 接到 compute server。和其他 NVMe architecture 的主要差異，只是把 ConnectX NIC 換成 BlueField NIC。

Atomic Claim 297/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0297

Claim: NVIDIA 在 1 月 CES 提出 inference memory hierarchy 中新的 intermediate storage「tier G3.5」，其技術為 NVMe。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 298/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0298

Claim: Tier G3.5 NVMe 位在 tier G3 DRAM 與下一層 shared storage 之間……
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 299/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0299

Claim: ……另一側是 tier G4 shared storage；原始 extraction 同時保留了數個 § placeholder。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 300/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0300

Claim: 這套平台先前稱為 ICMS，現在改稱 CMX；原文將其描述為透過 BlueField NICs 連接 compute server 的 storage server 架構之重新命名。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 301/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0301

Claim: 相較既有 NVMe architecture，原文稱唯一差異是把 Connect-X NIC 換成 BlueField NIC。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Source: Original NVIDIA ICMS blog in January, 2026 – updated and re-released on March 16, 2026.

STX

To expand the scope of CMX, NVIDIA also launched STX. STX is a reference rack architecture using Nvidia’s BF-4 based storage solution to complement VR compute racks. The reference architecture effectively specifies exactly how many drives, Vera CPUs, BF-4 DPUs, CX-9 NICs, and Spectrum-X switches are needed for a given cluster.

為擴大 CMX scope，NVIDIA 又推出 STX。STX 是一套 reference rack architecture，使用 Nvidia BF-4-based storage solution 搭配 VR compute rack。Reference architecture 幾乎直接規定一個 cluster 應該配置多少 drive、Vera CPU、BF-4 DPU、CX-9 NIC、Spectrum-X switch。

Atomic Claim 302/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0302

Claim: 為擴大 CMX 的範圍，NVIDIA 也推出 STX。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 303/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0303

Claim: STX 是 reference rack architecture，採用 Nvidia 的 BF-4 based storage solution，用來補充 VR compute racks。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 304/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0304

Claim: 這套 reference architecture 實際上會明確規定一個 cluster 需要多少 drives、Vera CPUs、BF-4 DPUs 與 CX-9 NICs。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 305/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0305

Claim: 特定 cluster 也需要 Spectrum-X switches。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

BF-4 in STX. Source: Nvidia, SemiAnalysis

STX 中的 BF-4。Source: Nvidia, SemiAnalysis

Atomic Claim 306/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0306

Claim: 來源標示為 Nvidia 與 SemiAnalysis。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Unlike the BF-4 in the VR NVL72, which consists of a Grace CPU and a single CX-9 NIC, the BF-4 in the STX reference design includes one Vera CPU, two CX-9 NICs, and two SOCAMM modules. Each STX box contains two BF-4 units, totaling two Vera CPUs, four CX-9 NICs, and four SOCAMM modules. For the whole STX rack, it has a total of 16 boxes, implying 32 Vera CPUs, 64 CX-9 NICs, and 64 SOCAMMs.

VR NVL72 裡的 BF-4 由一顆 Grace CPU + 一顆 CX-9 NIC 組成；STX reference design 的 BF-4 則包含一顆 Vera CPU、兩顆 CX-9 NIC、兩個 SOCAMM module。每個 STX box 放兩組 BF-4，所以每 box 合計兩顆 Vera、四顆 CX-9、四個 SOCAMM。整個 STX rack 有 16 個 box，也就是總共 32 顆 Vera CPU、64 顆 CX-9 NIC、64 個 SOCAMM。

Atomic Claim 307/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0307

Claim: 作為對照，VR NVL72 中的 BF-4 包含一顆 Grace CPU……
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 308/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0308

Claim: ……以及一顆 CX-9 NIC；相較之下，STX reference design 的 BF-4 包含一顆 Vera CPU、兩顆 CX-9 NIC 與兩個 SOCAMM module。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 309/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0309

Claim: 每個 STX box 含兩個 BF-4，合計兩顆 Vera CPU、四顆 CX-9 NIC 與四個 SOCAMM module。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 310/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0310

Claim: 整個 STX rack 有 16 個 box，因此推算共有 32 顆 Vera CPU、64 顆 CX-9 NIC 與 64 個 SOCAMM。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

STX Rack (left). Source: Nvidia, SemiAnalysis

STX Rack（左）。Source: Nvidia, SemiAnalysis

Atomic Claim 311/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0311

Claim: 來源標示為 Nvidia 與 SemiAnalysis。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

The STX announcement included a typical Nvidia show of strength where they named all major storage vendors as supporting STX, including AIC, Cloudian, DDN, Dell Technologies, Everpure, Hitachi Vantara, HPE, IBM, MinIO, NetApp, Nutanix, Supermicro, Quanta Cloud Technology (QCT), VAST Data and WEKA.

STX announcement 又上演了一次典型 Nvidia『秀肌肉』：直接把幾乎所有主要 storage vendor 都列成 STX supporter，包括 AIC、Cloudian、DDN、Dell Technologies、Everpure、Hitachi Vantara、HPE、IBM、MinIO、NetApp、Nutanix、Supermicro、Quanta Cloud Technology（QCT）、VAST Data、WEKA。

Atomic Claim 312/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0312

Claim: STX announcement 中，Nvidia 列出多家 storage vendor 支援 STX，包含 AIC、Cloudian、DDN、Dell Technologies、Everpure、Hitachi Vantara、HPE、IBM、MinIO、NetApp、Nutanix、Supermicro、QCT、VAST Data 與 WEKA。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Put together, BlueField-4, CMX, and STX represent NVIDIA’s broader effort to standardize how clusters are designed at the storage layer. NVIDIA has captured the compute and network layer, and is actively moving into the storage, software, and infrastructure operations layers over time.

合在一起看，BlueField-4、CMX、STX 代表 NVIDIA 正試圖把 cluster 在 storage layer 的設計方式也標準化。NVIDIA 已掌握 compute、network layer，現在正逐步往 storage、software、infrastructure operations layer 繼續延伸。

Atomic Claim 313/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0313

Claim: 合併來看，BlueField-4、CMX 與 STX 代表 NVIDIA 更廣泛地嘗試標準化 cluster storage layer 設計。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 314/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0314

Claim: NVIDIA 已經掌握 compute……
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 315/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0315

Claim: ……以及 network layer，並且正逐步進入 storage、software 與 infrastructure operations layer。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Now behind the paywall, we will share some more details on how all of this impacts the supply chain. Including beneficiaries of the LPX system, and the updated Kyber racks. We will also reveal a rack concept that Nvidia has yet to announce.

接下來 paywall 後，我們會分享這些變化對 supply chain 的更多影響，包括 LPX system、更新版 Kyber rack 的受益者，也會揭露一個 Nvidia 尚未正式宣布的 rack concept。

Atomic Claim 316/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0316

Claim: SemiAnalysis 表示，paywall 後將分享這些變化對 supply chain 的更多影響細節。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 317/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0317

Claim: 內容也包含 LPX system 與更新後 Kyber rack 的受益者。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 318/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0318

Claim: SemiAnalysis 也表示將揭露一個 Nvidia 尚未公布的 rack concept。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Feynman NVL1152 Networking Topologies

Within each Feynman Kyber rack, we tentatively assume double the bandwidth per logical GPU and double the NVLink Switch bandwidth to 28.8T and 57.6T respectively. Though Jensen, in the Financial Q+A the day after the GTC Keynote, characterized NVL1152 as “all CPO”, the key technical blog outlining the new rack form factors only strictly referenced CPO for rack to rack interconnect. We will discuss the potential topography for both options.

對每個 Feynman Kyber rack，我們暫時假設每 logical GPU bandwidth 加倍、NVLink Switch bandwidth 也加倍，分別到 28.8T、57.6T。雖然 Jensen 在 GTC Keynote 隔天 Financial Q+A 把 NVL1152 描述成『all CPO』，但介紹新 rack form factor 的核心 technical blog ↗ 嚴格來說只明確提到 rack-to-rack interconnect 使用 CPO。後面我們會分別討論兩種情境可能的 topology。

Atomic Claim 319/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0319

Claim: 在每個 Feynman Kyber rack 中，SemiAnalysis 暫時假設每個 logical GPU 的 bandwidth 會翻倍。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 320/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0320

Claim: 並假設 NVLink Switch bandwidth 分別提高到 28.8T 與 57.6T。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 321/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0321

Claim: Jensen 在 GTC keynote 隔日的 Financial Q+A 將 NVL1152 描述為「all CPO」，但 rack form-factor technical blog 僅明確提到 CPO 用於 rack-to-rack interconnect。
Frame: NARY_RELATION · Mode: ATTRIBUTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 322/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0322

Claim: SemiAnalysis 表示，後續會討論這兩種 option 的可能 topology。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

To double the scale-up bandwidth using copper interconnect, NVIDIA would have to achieve a per lane bandwidth of 448Gbit/s uni-di (and implemented with simultaneous bi-directional SerDes so that each physical channel carries 448G of RX and 448G of TX). However, this is a challenging feat as they would first have to prove the feasibility of 448Gb/s PAM4 SerDes at large volumes, then implement echo cancellation to achieve bidirectional bandwidth, which is in itself extremely difficult. We believe Nvidia is going for 448G uni-di only.

如果要靠 copper 把 scale-up bandwidth 再翻倍，NVIDIA 必須讓每 lane 達到 448Gbit/s uni-di；若還要 simultaneous bidirectional，等於每 physical channel 同時承載 448G RX + 448G TX。這非常困難，因為 Nvidia 不只要先證明 448Gb/s PAM4 SerDes 能大規模量產，還得再做 echo cancellation 才能達成 bi-directional bandwidth ↗，而這本身就極其困難。我們認為 Nvidia 實際上會走 448G uni-di。

Atomic Claim 323/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0323

Claim: 若要用 copper interconnect 將 scale-up bandwidth 翻倍，NVIDIA 必須做到每 lane 448Gbit/s uni-di；若採 simultaneous bi-directional，單一 physical channel 需同時承載 448G RX 與 448G TX。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 324/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0324

Claim: 這很困難：首先必須證明 448Gb/s PAM4 SerDes 能大規模量產，之後還要實作 echo cancellation 才能達成 bidirectional bandwidth。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 325/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0325

Claim: SemiAnalysis 判斷 Nvidia 會採 448G uni-di only。
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: PARTIAL
開啟逐條審核

Source: SemiAnalysis

Feynman could use in-rack optics, where switch blades are blind-mated to the midplane using optical connectors and thin fiber strands can be used to connect the optical connectors to the NVLink 8 Switches in place of flyover cables, but we believe this is very unlikely.

Feynman 理論上也可以在 rack 內使用 optics，例如 switch blade 透過 optical connector blind-mate 到 midplane，再用細 fiber strand 取代 flyover cable，把 optical connector 接到 NVLink 8 Switch；但我們認為這種設計非常不可能。

Atomic Claim 326/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0326

Claim: Feynman 可能採用 in-rack optics，讓 switch blade 透過 optical connector blind-mate 到 midplane。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 327/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0327

Claim: 細 fiber strands 可取代 flyover cable，用來把 optical connector 連到 NVLink 8 switch。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 328/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0328

Claim: 但 SemiAnalysis 認為上述 in-rack optics 方案非常不可能。
Frame: CLAIM_ONLY · Mode: EXPECTED · Mapping: PARTIAL
開啟逐條審核

Source: SemiAnalysis

For rack-to-rack interconnect, we explore two different topologies. The first is a two-layer CLOS network that is similar to the Oberon form factor, but with twice the bandwidth of each GPU and NVLink switch.

Rack-to-rack interconnect 方面，我們探索兩種 topology。第一種是類似 Oberon form factor 的 two-layer CLOS network，但每顆 GPU、NVLink switch bandwidth 都加倍。

Atomic Claim 329/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0329

Claim: 對 rack-to-rack interconnect，SemiAnalysis 探討兩種不同 topology。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 330/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0330

Claim: 第一種是兩層 CLOS network，類似 Oberon form factor，但每顆 GPU 與 NVLink switch bandwidth 都是兩倍。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Source: SemiAnalysis

The second is a reconfigurable dragonfly topology using OCS switches to connect the 8 racks. The number of OCS ports required for this topology remains tentative.

第二種是 reconfigurable dragonfly topology，用 OCS switch 連接 8 個 rack；這種 topology 最後需要多少 OCS port，目前仍是 tentative。

Atomic Claim 331/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0331

Claim: 第二種是可重組的 dragonfly topology，使用 OCS switch 連接 8 個 rack。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 332/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0332

Claim: 這種 topology 所需的 OCS port 數量目前仍是暫定。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Source: SemiAnalysis

GTC 2026 Supply Chain Implications

Here, we will discuss our findings on where we see there are big changes in content for the supply chain coming out of all these announcements at GTC.

下面會整理這次 GTC announcement 中，我們認為對 supply-chain content 變化最大的幾個地方。

Atomic Claim 333/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0333

Claim: 接下來 SemiAnalysis 將討論這些 GTC announcement 對 supply chain content 造成的重大變化。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

AlphaWave 112G Serdes in LP30

It may surprise readers that Qualcomm has IP in the Groq LPU 3 chip! More specifically it is AlphaWave, which Qualcomm acquired last year, that is providing the 112G SerDes for Groq’s C2C. AlphaWave was selected as the only IP provider that has high speed SerDes for Samsung Foundry. It was AlphaWave’s SerDes that caused issues for Groq LPU 2. Alphawave will continue to be used for the LP35, but Nvidia will of course use their own NVLink SerDes IP from LP40 when it transitions back to TSMC.

讀者可能會很意外：Groq LPU 3 裡竟然有 Qualcomm IP。更精確地說，是 Qualcomm 去年收購的 AlphaWave，為 Groq C2C 提供 112G SerDes。AlphaWave 當初被選中，是因為它是少數在 Samsung Foundry 擁有 high-speed SerDes 的 IP provider。LPU 2 出問題的，也正是 AlphaWave SerDes。LP35 仍會繼續用 AlphaWave；但到 LP40 轉回 TSMC 後，Nvidia 當然會改用自己的 NVLink SerDes IP。

Atomic Claim 334/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0334

Claim: 值得注意的是，Qualcomm 的 IP 實際存在於 Groq LPU 3 晶片中。
Frame: ATTRIBUTE · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 335/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0335

Claim: 更具體地說，去年被 Qualcomm 收購的 AlphaWave，為 Groq 的 C2C 提供 112G SerDes。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 336/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0336

Claim: AlphaWave 被選為 Samsung Foundry 高速 SerDes 的唯一 IP provider。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 337/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0337

Claim: AlphaWave 的 SerDes 曾造成 Groq LPU 2 的問題。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 338/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0338

Claim: LP35 仍會繼續使用 Alphawave。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 339/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0339

Claim: 當 LP40 轉回 TSMC 後，Nvidia 將使用自家的 NVLink SerDes IP。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

LPX PCB

Next, we mentioned that a very high spec PCB is required for the LPX compute tray. We estimate that each compute tray main board PCB will carry $7k ASP. The suppliers for this are Victory Giant and WUS. Of course, there are several other PCB modules in the compute tray, but they do not need a high spec. Nvidia is continuing with their cable-less philosophy similar to the Vera Rubin compute tray which requires a lot of board-to-board connectors, which brings us to the next big beneficiary.

再來，前面提到 LPX compute tray 需要非常高規格 PCB。我們估計每個 compute tray main-board PCB ASP 約 $7k，supplier 是 Victory Giant、WUS。Compute tray 內當然還有其他 PCB module，但不需要同等級 specification。Nvidia 也延續 Vera Rubin compute tray 類似的 cableless philosophy，因此需要大量 board-to-board connector，這就帶到下一個主要受益者。

Atomic Claim 340/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0340

Claim: LPX compute tray 需要非常高規格的 PCB。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 341/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0341

Claim: SemiAnalysis 估計，每個 compute tray main-board PCB 的 ASP 約為 7,000 美元。
Frame: ATTRIBUTE · Mode: ESTIMATED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 342/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0342

Claim: 原文指出，該 PCB 的供應商是 Victory Giant 與 WUS。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 343/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0343

Claim: compute tray 中還有數個其他 PCB module，但這些並不需要高規格。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 344/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0344

Claim: Nvidia 持續採用類似 Vera Rubin compute tray 的 cable-less philosophy，因此需要大量 board-to-board connectors。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Cables and Connectors: Amphenol Continues to Benefit

For the LPX, Amphenol will be a beneficiary for all the connectors for the backplane. Each LPX node requires 16 80DP Paladin connectors for the backplane. There are also board to board connectors required to connect all the various modules within the tray: the main LPU board with the host CPU module and the OSFP/QSFP modules that sit below the CPU module, the front-end NIC module, and the management module. Amphenol will supply the cable backplane too which is 8,160 DP per rack.

LPX 方面，Amphenol 會受益於 backplane 與 tray 內大量 connector。每個 LPX node 需要 16 個 80DP Paladin connector 連 backplane。此外，tray 內各 module 互連也需要 board-to-board connector：main LPU board 要連 host CPU module、位於 CPU module 下方的 OSFP/QSFP module、frontend NIC module、management module。Amphenol 也會供應 cable backplane，整 rack 規模達 8,160 DP。

Atomic Claim 345/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0345

Claim: 在 LPX 中，Amphenol 會受益於 backplane 所需的 connector content。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 346/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0346

Claim: 每個 LPX node 的 backplane 需要 16 個 80DP Paladin connectors。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 347/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0347

Claim: tray 內也需要 board-to-board connectors 連接各 module，例如主 LPU board 與 host CPU module……
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 348/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0348

Claim: ……另外還包括位於 CPU module 下方的 OSFP／QSFP module、front-end NIC module與 management module。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 349/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0349

Claim: Amphenol 也會供應 cable backplane，每 rack 為 8,160 DP。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

NVL288 System

For the Vera Rubin Ultra NVL288 System that we discussed above, we would say the cable backplane return for Kyber. If Rubin Ultra is deployed in such a form factor – each of the Rubin Ultra GPUs will have a scale-up bandwidth of 14.4Tbit/s uni-di, requiring 144 DPs of cables to connect to the NVSwitches. 144 DPs times 288 GPUs means a total of 41,472 DPs to connect this larger world size domain. This is a lot of cables, so it is more of an upper bound of how much cable content could be used here. If there is oversubscription or if the inter-rack connection is made through the switches – it is possible fewer DPs would be needed.

前面討論的 Vera Rubin Ultra NVL288，如果真的採這個 form factor，我們認為 Kyber 會重新需要大量 cable backplane。每顆 Rubin Ultra GPU scale-up bandwidth 14.4Tbit/s uni-di，需要 144 DP cable 連 NVSwitch；144 DP × 288 GPU = 41,472 DP，才能連完整個 larger world-size domain。這是非常巨量的 cable，因此比較像 cable content 的 upper-bound case。若 architecture 有 oversubscription，或 rack-to-rack connection 改經 switch，實際所需 DP 可能更少。

Atomic Claim 350/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0350

Claim: 對前述 Vera Rubin Ultra NVL288 system，SemiAnalysis 認為 Kyber 會重新採用 cable backplane。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 351/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0351

Claim: 若 Rubin Ultra 以此 form factor 部署，每顆 Rubin Ultra GPU 將提供 14.4Tbit/s uni-di scale-up bandwidth，並需要 144 DP cable 連到 NVSwitch。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 352/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0352

Claim: 144 DP × 288 顆 GPU，代表連接這個較大 world-size domain 共需要 41,472 DP。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 353/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0353

Claim: 這代表大量 cable，因此 41,472 DP 更適合作為此架構可能 cable content 的上限。
Frame: CLAIM_ONLY · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 354/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0354

Claim: 如果採 oversubscription，或 inter-rack connection 透過 switch 完成，所需 DP 數量可能更少。
Frame: CLAIM_ONLY · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟逐條審核

FIT Joining the Party

Backplane cable cartridge and Paladin connector demand is so strong that Amphenol cannot keep up with supply. Amphenol has now completed licensing of the VR NVL72 backplane cable cartridge as well as Paladin HD connectors to FIT, who can now manufacture these components. This has been in the works for a long time but is finally settled. Amphenol will earn licensing fees from FIT’s sales of these licensed components.

Backplane cable cartridge、Paladin connector demand 強到 Amphenol 自己已供不應求。Amphenol 現在已完成把 VR NVL72 backplane cable cartridge、Paladin HD connector 授權給 FIT，FIT 可以自行製造。這件事談了很久，現在終於確定；Amphenol 會從 FIT 銷售這些 licensed component 中收取 licensing fee。

Atomic Claim 355/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0355

Claim: Backplane cable cartridge 與 Paladin connector demand 強到 Amphenol 無法完全供應。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 356/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0356

Claim: Amphenol 已完成將 VR NVL72 backplane cable cartridge 與 Paladin HD connectors 授權給 FIT，FIT 現在可以製造這些元件。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 357/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0357

Claim: 這項安排已推進很長一段時間，如今終於敲定。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 358/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0358

Claim: Amphenol 將從 FIT 銷售這些授權元件中取得 licensing fee。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Kyber Voronoi – Another FIT Win?

The Kyber midplane will utilize many 8×19 DP connectors to interface with the compute trays at the front of the rack, and to the switch blades in the back of the rack.

Kyber midplane 會使用大量 8×19 DP connector，前側連 compute tray，後側連 switch blade。

Atomic Claim 359/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0359

Claim: Kyber midplane 將使用大量 8×19 DP connector，連接 rack 前方的 compute tray 與後方 switch blade。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

For Kyber, Nvidia is now in the driver’s seat when it comes to IP and they have designed a proprietary connector spec named Voronoi, so it will no longer be the Amphenol Paladin connector. There are three vendors bidding for the project: FIT, Molex and Amphenol. FIT appears to be leading the market for these connectors, but Amphenol is reportedly also working together closely with FIT to manufacture the connectors. The design and implementation of Voronoi remains in flux, but both FIT and Amphenol will need to ramp significant production volume with the specification licensed from Nvidia.

Kyber connector IP 方面，Nvidia 現在已掌握主導權，並設計一套 proprietary spec，名稱叫 Voronoi，因此不再使用 Amphenol Paladin connector。目前有三家 vendor 競標：FIT、Molex、Amphenol。FIT 看起來暫時領先，但據了解 Amphenol 也和 FIT 密切合作，共同生產 connector。Voronoi design、implementation 仍在變動，但 FIT、Amphenol 都必須依 Nvidia 授權的 specification，準備 ramp 非常大的 production volume。

Atomic Claim 360/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0360

Claim: 在 Kyber 的 IP 設計上，Nvidia 現在居於主導地位。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 361/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0361

Claim: Nvidia 設計了名為 Voronoi 的 proprietary connector spec，因此不再沿用 Amphenol 的 Paladin connector。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 362/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0362

Claim: 目前有三家 vendor 競標此專案：FIT、Molex 與 Amphenol。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 363/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0363

Claim: FIT 目前看起來在這類 connectors 的競爭中居於領先。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 364/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0364

Claim: 據報導，Amphenol 也正與 FIT 密切合作製造這些 connector。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 365/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0365

Claim: Voronoi 的設計與實作目前仍在變動中。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 366/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0366

Claim: FIT 與 Amphenol 都需要依照 Nvidia 授權的 specification，大幅提升 production volume。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

The midplane, switch tray and compute tray all feature female side connectors which will require the use of a spring-loaded male part that protect the pins and interface between both sides. The density of these connectors will ultimately be much higher than Amphenol’s Paladin connectors.

Midplane、switch tray、compute tray 都使用 female-side connector，因此需要一個 spring-loaded male component 介於兩側，保護 pin 並完成 interface。這些 connector 最後的 density 會明顯高於 Amphenol Paladin。

Atomic Claim 367/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0367

Claim: midplane、switch tray 與 compute tray 都採 female-side connector，因此需要 spring-loaded male part 保護 pins 並連接兩側。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 368/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0368

Claim: 這些新 connector 最終的 density 將遠高於 Amphenol 的 Paladin connector。
Frame: COMPARISON · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

More is available to our institutional subscribers sales@semianalysis.com.

更多內容提供給 institutional subscriber，洽詢 sales@semianalysis.com。

Atomic Claim 369/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0369

Claim: 更多內容提供給 SemiAnalysis institutional subscribers。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Mid-board Optics – Nvidia’s War on Pluggables

Interestingly, the Kyber rack exhibited at the GTC 2026 show floor is missing OSFP cages for scale-out networking. Instead, we only see 4x MPO ports from each compute tray. This design has effectively taken key pluggable transceiver items (driver, TIA, etc.) other than the DSP and put them on a Midboard Optical Module (MBOM) which then connects to the PCB via a land grid array (LGA) socket. Two CX-9s share one MBOM, which then connects to the MPO faceplate via a short fiber connection. The MBOM provides two MPO ports at 2x800G each for 1.6T of total connectivity.

有趣的是，GTC 2026 show floor 展示的 Kyber rack 沒有 scale-out networking 用的 OSFP cage；每個 compute tray 前面看到的反而只有 4 個 MPO port。這套 design 等於把 pluggable transceiver 的主要元件——driver、TIA 等，DSP 除外——移到 Midboard Optical Module（MBOM），再透過 land-grid-array（LGA）socket 接到 PCB。兩顆 CX-9 共用一個 MBOM，之後用短 fiber connection 接到 MPO faceplate。每個 MBOM 提供兩個 MPO port，每 port 2×800G，總 connectivity 1.6T。

Atomic Claim 370/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0370

Claim: GTC 2026 展場展示的 Kyber rack 缺少供 scale-out networking 使用的 OSFP cages。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 371/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0371

Claim: 相反地，每個 compute tray 只看到 4 個 MPO ports。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 372/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0372

Claim: 此設計把 pluggable transceiver 中除 DSP 以外的若干關鍵元件移出，包括 driver 與原文未解析的 § placeholder……
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 373/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0373

Claim: ……並將這些元件放到 Midboard Optical Module 上，再透過 land-grid-array socket 連接到 PCB。
Frame: RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Atomic Claim 374/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0374

Claim: 兩顆 CX-9 共用一個 MBOM，MBOM 再透過短距離 fiber 連到 MPO faceplate。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 375/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0375

Claim: MBOM 提供兩個 MPO ports，每個為 2×800G，總 connectivity 為 1.6T。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

4 MPO ports on the left, rather than OSFP cages. Source: Nvidia, SemiAnalysis

左側 4 個 MPO port，而不是 OSFP cage。Source: Nvidia, SemiAnalysis

Atomic Claim 376/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0376

Claim: 左側配置 4 個 MPO ports，而不是 OSFP cages。
Frame: COMPARISON · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 377/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0377

Claim: 來源標示為 Nvidia 與 SemiAnalysis。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

The use of MBOM would block the use of any form of pluggable transceiver or AEC, and naturally hyperscalers are saying “CP-Hell No” to that idea and are continuing to push for an OSFP cage so they can continue using pluggables.

但使用 MBOM 等於直接封死 pluggable transceiver、AEC 的可能性，hyperscaler 當然對此說『CP-Hell No』，仍持續推動保留 OSFP cage，讓它們可以繼續使用 pluggable。

Atomic Claim 378/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0378

Claim: 採用 MBOM 後，任何形式的 pluggable transceiver 或 AEC 都無法使用。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 379/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0379

Claim: Hyperscaler 對 MBOM 方案明顯反對，並持續要求保留 OSFP cage，以便繼續使用 pluggable。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

It is important to point out that many aspects of the Kyber design are still in flux and there could still be a number of design changes before Kyber racks are actually deployed. After all – the change from the four canister design to a two compute tray canister + one switch blade bank is already a huge change.

需要強調，Kyber design 很多部分仍在變，真正 deployment 前還可能有不少 design change。光是從原本四個 canister，改成兩個 compute-tray canister + 一個 switch-blade bank，就已經是非常大的變動。

Atomic Claim 380/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0380

Claim: 必須注意，Kyber design 的許多部分目前仍在變動中。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 381/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0381

Claim: 在 Kyber rack 真正部署前，仍可能出現多項 design change。
Frame: ATTRIBUTE · Mode: HYPOTHETICAL · Mapping: COMPLETE
開啟逐條審核

Atomic Claim 382/382 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0382

Claim: 畢竟，從四個 canister design 改成兩個 compute tray canister 加上一個 switch-blade bank，本身就已是很大的改動。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟逐條審核

Research Vault

探索

coverage_before_refresh__2026-03-24_nvidia-the-inference-kingdom-expands

SA Article Coverage Review · 2026-03-24_nvidia-the-inference-kingdom-expands

Coverage Summary

Missing Claim Notes

SA Full Text + Translation + Atomic Claims

Nvidia – The Inference Kingdom Expands

Groq

LPU chip

SRAM and Memory Hierarchy

GPU and LPU Integration: Attention FFN Disaggregation (AFD)

Speculative Decoding

LPX Rack System

LPX Compute Tray

LPU Network

Intra-Tray Topology

Inter-node/Intra-rack

Inter-rack

Nvidia’s CPO Roadmap

Oberon and Kyber Updates, Larger World Sizes Introduced, More Networking Updates

Rubin Ultra NVL288

Rubin Ultra NVL576

Feynman

Vera ETL256

CMX and STX

CMX

STX

Feynman NVL1152 Networking Topologies

GTC 2026 Supply Chain Implications

AlphaWave 112G Serdes in LP30

LPX PCB

Cables and Connectors: Amphenol Continues to Benefit

NVL288 System

FIT Joining the Party

Kyber Voronoi – Another FIT Win?

Mid-board Optics – Nvidia’s War on Pluggables

關係圖譜

目錄