Phase C–E 審查快照

日常閱讀請使用原 SA Coverage 入口，該頁已保留人工畫線並更新最新 Claim。

SA Article Coverage Review · 2026-03-24_nvidia-the-inference-kingdom-expands

Coverage Summary

Source: 開啟原始 SA 文章
Atomic Claims: 503
Source blocks: 227
Blocks with ≥1 Atomic Claim: 103
Blocks without Atomic Claim: 124
Unplaced Claims: 0

Coverage Review

本版依已封存的 Phase C–E 審查結果產生；Claim 按正式 Evidence 位置插入。
若該英文段落已有繁中翻譯 cache，翻譯只會作為淡色閱讀輔助顯示；不會進入 source、Claim provenance 或 Graph。
沒有 Claim callout 的段落不一定有問題；若內容重要且應形成知識，請記到 Missing Claim Notes。

Missing Claim Notes

本版已完成使用者委託的來源與語意檢查。後續人工意見可另行補充；所有決策以正式 Ledger 為準。

SA Full Text + Translation + Atomic Claims

Nvidia – The Inference Kingdom Expands

Source: Nvidia

At GTC 2026, Nvidia delivered an event packed full of ground breaking announcements. Nvidia’s pace of innovation is not showing any signs of slowing, as they introduced three entirely new systems this year: Groq LPX, Vera ETL256, and STX. Also announced were updates to Nvidia’s Kyber rack architecture system, CPO making its debut for scale-up networking with the unveiling of the Rubin Ultra NVL576 and Feynman NVL1152 multi-rack systems. Early hints on Feynman’s architecture was also a key topic. A Jensen callout for InferenceX during the keynote was a highlight. ↗

GTC 2026，Nvidia 帶來一場塞滿重大 announcement 的發表會。Innovation pace 完全沒有放慢，今年一次推出三套全新 system：Groq LPX、Vera ETL256、STX。同時還公布 Kyber rack architecture 更新，並讓 CPO 首次進入 scale-up networking，推出 Rubin Ultra NVL576 與 Feynman NVL1152 multi-rack system；Feynman architecture 的早期線索也是重點之一。Jensen 在 keynote 上特別點名 InferenceX，則是另一個亮點 ↗。

Atomic Claim 1/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0002::SPLIT01

Claim: 截至 GTC 2026，Nvidia 的創新速度沒有顯示放緩跡象。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 2/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0002

Claim: Nvidia 推出了 Groq LPX。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 3/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0006

Claim: At GTC 2026, Nvidia announced the roadmap: CPO 首度被用於 Rubin Ultra NVL576 多機櫃系統的 scale-up networking。
Frame: NARY_RELATION · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 4/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0004

Claim: Nvidia 推出了 STX。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 5/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0003

Claim: Nvidia 推出了 Vera ETL256。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 6/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0005

Claim: At GTC 2026, Nvidia announced the roadmap: Nvidia 公布了 Kyber rack 架構系統的更新。
Frame: RELATION · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 7/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0007

Claim: At GTC 2026, Nvidia announced the roadmap: CPO 首度被用於 Feynman NVL1152 多機櫃系統的 scale-up networking。
Frame: NARY_RELATION · Mode: ATTRIBUTED · Mapping: PARTIAL
開啟本版 Claim 與證據

This is our GTC 2026 recap, and we will address many of the key questions that have been left unanswered by Nvidia. Specifically, we will go through the LPX rack and LP30 chip and explain how attention and feed forward network disaggregation (AFD) works; more details on the various rack architectures behind NVL144, NVL576, and NVL1152 and clarify just how much optics will be inserted as well as the rationale behind the dense Vera ETL256. The next generation Kyber rack had some big updates and some hidden details.

這篇是我們的 GTC 2026 recap，也會回答許多 Nvidia 留下的關鍵問題。具體來說，我們會拆解 LPX rack、LP30 chip，說明 attention 與 feed-forward network disaggregation（AFD）如何運作；進一步說明 NVL144、NVL576、NVL1152 背後不同 rack architecture，到底會插入多少 optics，以及 dense Vera ETL256 的設計邏輯。下一代 Kyber rack 也有幾項重大更新，以及一些 Nvidia 沒明說的細節。

Groq

First up is the Groq LPU. One of the most significant recent events in AI infrastructure was Nvidia’s “acquisition” of Groq. Strictly speaking, Nvidia paid Groq $20B to license their IP and hire most the team. This functions almost as an acquisition, though its structure technically falls short of it being legally considered as one, thereby simplifying or obviating the need for regulatory approvals. Given Nvidia’s market share, if this transaction were structured as a full acquisition and were put to anti-trust review, such a transaction would likely not go through. The other benefit is that it avoids a drawn-out transaction closing process. Nvidia got instant access to Groq’s IP and people. This is why, less than four months after the deal was announced, Nvidia already has a system concept that is being integrated into the Vera Rubin inference stack.

先從 Groq LPU 開始。近期 AI infrastructure 最重要事件之一，就是 Nvidia 對 Groq 的『acquisition』。嚴格來說，Nvidia 支付 Groq $20B，是為了 license Groq IP 並雇用大部分團隊。實質效果幾乎等於收購，但法律結構刻意沒有做到完整 acquisition，因此可簡化、甚至避開 regulatory approval。考量 Nvidia market share，如果交易真的以完整收購形式送 anti-trust review，很可能根本過不了。另一個好處是避免漫長 closing process；Nvidia 幾乎立即取得 Groq IP 與人員。這也解釋了為什麼交易宣布不到四個月，Nvidia 就已經拿出一個 system concept，準備整合進 Vera Rubin inference stack。

Atomic Claim 8/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0015::SPLIT01

Claim: SemiAnalysis states: Nvidia 與 Groq 的這項交易在實質效果上幾乎等同收購。
Frame: NARY_RELATION · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 9/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0020

Claim: 因此，在交易宣布不到四個月後，Nvidia 已經提出一個正整合進 Vera Rubin inference stack 的系統概念。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 10/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0014

Claim: Given Nvidia’s market share, SemiAnalysis judges that a hypothetical full acquisition of Groq would likely not pass antitrust review.
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 11/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0019::SPLIT02

Claim: Nvidia 立即取得了 Groq 的人才資源。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 12/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0014::SPLIT02

Claim: 在這筆 200 億美元交易中，Nvidia 聘用 Groq 大部分團隊成員。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 13/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0018

Claim: 另一項好處是，這種交易結構可避免冗長的交易交割流程。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 14/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0015::SPLIT02

Claim: SemiAnalysis states: Nvidia 與 Groq 的這項交易在法律結構上不足以被視為正式收購。
Frame: NARY_RELATION · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 15/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0015::SPLIT03

Claim: SemiAnalysis states: 由於這項交易在法律上不構成正式收購，因此可簡化、甚至免除監管核准需求。
Frame: CLAIM_ONLY · Mode: ATTRIBUTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 16/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0019::SPLIT01

Claim: Nvidia 立即取得了 Groq IP 的存取權。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 17/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0014::SPLIT01

Claim: 在這筆 200 億美元交易中，Nvidia 取得 Groq IP 授權。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Let’s now go through a refresher on the LPU architecture to see how Groq’s LPU complements Nvidia’s GPU. For more details see our original Groq piece. ↗ The premise from that piece remains unchanged: the standalone Groq LPU system is not economical for serving tokens at scale, but it can serve tokens very quickly which can demand a large market premium. This is the premise behind how LPU fits into a disaggregated decode system.

接著快速複習 LPU architecture，看 Groq LPU 如何和 Nvidia GPU 互補。更多細節可參考我們原本的 Groq 文章 ↗。當時的核心論點完全沒變：standalone Groq LPU system 若拿來大規模 serving token，economics 並不好；但它可以把 token 產生得非常快，因此有機會收取很高 market premium。這正是 LPU 被放進 disaggregated decode system 的基本邏輯。

Atomic Claim 18/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0023

Claim: 先前文章的核心看法仍未改變：獨立運作的 Groq LPU 系統，在大規模 token serving 下並不具經濟性。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 19/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0024::SPLIT02

Claim: Groq LPU 的極快 token serving 能力可以帶來很高的市場溢價。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 20/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0024::SPLIT01

Claim: 獨立運作的 Groq LPU system 可以非常快速地提供 tokens。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

LPU chip

Groq’s first and only publicly announced LPU architecture was detailed in their ISCA 2020 paper. Unlike typical hardware architectures connecting many general-purpose cores, Groq re-organized the architecture into groups of single-purpose units connecting to other groups of different purposes, and they named the groups “slices.” Between functional units are streaming registers, scratchpad SRAM for functional units to pass data to each other. Groq opted for single-level scratchpad SRAM instead of multi-level memory hierarchy to make the hardware execution deterministic.

Groq 第一代、也是唯一公開完整介紹過的 LPU architecture，來自 ISCA 2020 paper。和一般把很多 general-purpose core 互連的 hardware architecture 不同，Groq 把 architecture 重新組成多組 single-purpose unit，再讓不同用途的 group 互相連接，並把這些 group 稱為『slice』。Functional unit 之間使用 streaming register，也就是 scratchpad SRAM，讓不同 unit 交換 data。Groq 刻意採 single-level scratchpad SRAM，而不是 multi-level memory hierarchy，目的是讓 hardware execution 具有 deterministic 特性。

Atomic Claim 21/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0023

Claim: 各功能單元之間透過串流暫存器與 scratchpad SRAM 傳遞資料。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 22/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0028

Claim: Groq 選擇單層 scratchpad SRAM，而不是多層記憶體階層，以讓硬體執行具備 deterministic 特性。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 23/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0022

Claim: 不同於以大量通用核心互連的典型硬體架構，Groq 將架構重組為不同用途的單功能單元群組，並將這些群組稱為「slices」。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 24/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0026

Claim: Groq 第一個、也是唯一公開宣布過的 LPU 架構，詳述於其 ISCA 2020 論文。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Concretely, LPU architecture has VXM slices for vector operations, MEM slices for loading/storing data, SXM slices for tensor shape manipulation, and MXM slices for performing matrix multiplication. Spatially, the slices are laid out horizontally, allowing the data to stream horizontally. Within a slice, instructions are pumped vertically across units. Conceptually, LPU resembles a systolic array that pumps instructions vertically and data horizontally.

具體來說，LPU architecture 有負責 vector operation 的 VXM slice、load／store data 的 MEM slice、tensor shape manipulation 的 SXM slice，以及執行 matrix multiplication 的 MXM slice。空間布局上，這些 slice 水平排列，data 因此沿水平方向 streaming；每個 slice 內 instruction 則垂直穿過各 unit。概念上，LPU 很像一個 instruction 垂直流動、data 水平流動的 systolic array。

Atomic Claim 25/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0030

Claim: In the described Groq LPU architecture, functional slices are arranged horizontally so data streams horizontally.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 26/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0031

Claim: Within each Groq LPU functional slice, instructions travel vertically across its units.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 27/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0025

Claim: 具體而言，LPU 架構包含用於向量運算的 VXM slices。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 28/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0027

Claim: 具體而言，LPU 架構包含用於張量形狀操作的 SXM slices。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 29/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0026

Claim: 具體而言，LPU 架構包含用於資料載入與儲存的 MEM slices。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 30/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0032

Claim: 概念上，LPU 類似 systolic array：instructions 沿垂直方向傳送，而資料沿水平方向流動。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 31/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0028

Claim: 具體而言，LPU 架構包含用於矩陣乘法的 MXM slices。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 32/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-144

Claim: The Groq streaming architecture diagram labels 144 instruction-dispatch paths.
Frame: ATTRIBUTE · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: Groq, SemiAnalysis

The data flow and instruction flow design requires fine-grained pipelining to achieve high performance. Since LPU architecture makes computation deterministic, the compiler can aggressively schedule and overlap instructions to hide latency. The LPU’s use of high bandwidth SRAM and aggressive pipelining are the two main factors that enable LPU’s low latency.

這種 data flow、instruction flow design 要靠非常 fine-grained pipelining 才能得到高 performance。因為 LPU architecture 讓 computation 變成 deterministic，compiler 可以非常 aggressive 地 schedule、overlap instruction，把 latency 隱藏起來。High-bandwidth SRAM + aggressive pipelining，就是 LPU low latency 的兩大核心來源。

Atomic Claim 33/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0035::REPAIR01

Claim: High-bandwidth SRAM is one of the two main factors enabling Groq LPU low latency.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 34/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0033

Claim: The described Groq LPU data/instruction-flow architecture requires fine-grained pipelining for high performance.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 35/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0034

Claim: 由於 LPU 架構讓運算具備 deterministic 特性，compiler 可以更積極地排程並重疊 instructions，以隱藏 latency。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 36/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0036

Claim: 積極的 pipelining 也是促成 LPU 低 latency 的兩大主要因素之一。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

LPU gen 1 was designed on a legacy Global Foundries 14nm process, with Marvell responsible for the chip’s physical design. This was a much more mature node compared to peers when it taped out in 2020, with the incumbent AI chip platforms mostly on TSMC’s N7 platform. This made sense for an early product focused on proving out Groq’s architecture and bringing its inference-centric design to market. The 14nm node was mature, relatively well understood, and suitable for an initial chip where architectural differentiation mattered more than pushing its silicon to the leading edge.

LPU Gen1 採較舊的 GlobalFoundries 14nm，由 Marvell 負責 chip physical design。2020 tapeout 時，主流 AI chip platform 大多已在 TSMC N7，因此 14nm 明顯是成熟很多的 node。不過對 Groq 早期產品來說這是合理選擇：重點是先證明 architecture、把 inference-centric design 推到市場，而不是硬上 leading-edge silicon。14nm 成熟、理解充分，對一顆 architecture differentiation 比 transistor scaling 更重要的 first chip 很適合。

Atomic Claim 37/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0037

Claim: 第一代 LPU 採用較舊的 Global Foundries 14nm 製程設計。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 38/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0038

Claim: Marvell performed the physical design of Groq LPU Gen 1.
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 39/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0039

Claim: When Groq LPU Gen 1 taped out in 2020, its 14nm process was more mature than the TSMC N7 process used by most incumbent AI-chip platforms.
Frame: COMPARISON · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 40/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0038

Claim: SemiAnalysis considers a mature process node reasonable for Groq LPU Gen 1 because the initial product prioritized proving the architecture.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 41/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0042

Claim: 14nm 製程已相當成熟、業界也較熟悉，因此適合初代晶片；對這類產品而言，架構差異化比把 silicon 推到最先進節點更重要。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

One of the selling points is that the chip can be manufactured and packaged entirely in the United States compared to their competitors being heavily reliant on the Asia semiconductor supply chain: logic and packaging in Taiwan, with HBM from Korea.

它的一個賣點是 chip 可以完全在美國製造、封裝；競爭對手則高度依賴亞洲 semiconductor supply chain：logic、packaging 在台灣，HBM 來自韓國。

Atomic Claim 42/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0043

Claim: 競爭對手的加速器供應鏈仰賴來自韓國的 HBM。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 43/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0040

Claim: Groq LPU Gen 1 can be manufactured entirely in the United States.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 44/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0042

Claim: 競爭對手的加速器供應鏈高度依賴台灣的邏輯晶片製造與 packaging。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 45/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0041

Claim: Groq LPU Gen 1 can be packaged entirely in the United States.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Since then, Groq’s roadmap has stalled due to execution, with no LPU 2 having been shipped. This leaves the Groq LPU looking even more dated against competing roadmaps. What was once a meaningful but still manageable node disadvantage versus 7nm-era peers has widened into a far sharper gap, with all leading accelerator platforms now moving onto 3nm-class processes in 2026.

但後來 Groq roadmap 因 execution 卡住，LPU 2 一直沒有真正出貨，讓 Groq LPU 相較競爭 roadmap 看起來越來越老。原本對 7nm-era peer 而言還只是明顯、但可控制的 node disadvantage，到 2026 年所有 leading accelerator platform 都轉向 3nm-class process 後，差距已經拉得非常大。

Atomic Claim 46/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0048

Claim: 2026 年所有領先的加速器平台都正轉向 3nm 等級製程。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 47/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0047

Claim: Groq LPU Gen 1’s process disadvantage relative to competing accelerators widened after its original gap against 7nm-era peers.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 48/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0046

Claim: LPU 2 至今沒有實際出貨。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 49/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0047

Claim: 這也使 Groq LPU 相較競爭對手的 roadmap 顯得更加落後。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 50/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0045

Claim: 此後，Groq 的 roadmap 因執行問題而陷入停滯。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

The follow on Groq LPU 2 was designed for Samsung Foundry’s SF4X node, specifically at Samsung’s Austin fab, allowing them to extend the pitch that Groq is fabricated domestically in the USA. Samsung would also provide support for the back-end design. The choice of Samsung was driven by favorable terms / investment, with Samsung Foundry struggling to find customers for its advanced nodes and missing out on an AI logic customer. Unsurprisingly, Samsung was a key investor in Groq’s subsequent Series D in August 2024, and most recently in September 2025 before the Nvidia “acquisition.”

後續 Groq LPU 2 原本設計給 Samsung Foundry SF4X，且指定 Samsung Austin fab，讓 Groq 可以延續『美國本土製造』的 pitch。Samsung 也會支援 back-end design。選 Samsung 的原因是條件與投資都很優惠；Samsung Foundry 當時先進 node 缺 customer，也一直沒拿到 AI logic 大客戶。毫不意外，Samsung 後來成為 Groq 2024 年 8 月 Series D 的重要 investor，2025 年 9 月、Nvidia『acquisition』前也再次投資。

Atomic Claim 51/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0050

Claim: Groq LPU 2 原規劃在 Samsung Austin fab 生產。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 52/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0053::SPLIT02

Claim: Samsung 在 2025 年 9 月、Nvidia 對 Groq 的「收購」前再次投資 Groq。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 53/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0053::SPLIT01

Claim: Samsung 是 Groq 2024 年 8 月 Series D 的重要投資人。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 54/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0051

Claim: 採用 Samsung Austin fab，讓 Groq 得以維持「美國本土製造」的產品定位。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 55/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0052::SPLIT02

Claim: Samsung Foundry 當時正苦於替先進製程節點尋找客戶。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 56/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0051

Claim: Samsung would also support back-end design of the second-generation Groq LPU.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 57/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0049

Claim: Groq LPU 2 原設計採用 Samsung Foundry 的 SF4X 製程。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 58/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0052::SPLIT03

Claim: Samsung Foundry 當時未能取得 AI logic 客戶。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 59/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0052::SPLIT01

Claim: Groq 選擇 Samsung Foundry 的原因包括有利的商業條件與投資關係。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

However, the Groq LPU 2 was never productized because of design issues. The C2C SerDes on the chip couldn’t hit the advertised 112G speed which caused the design to malfunction, as we detailed long ago in the Accelerator model ↗. The third generation Groq LPU is the one that Nvidia will be productizing.

但 Groq LPU 2 最後因 design issue 從未 productize。Chip 上的 C2C SerDes 達不到 advertised 112G speed，導致整體 design 無法正常運作；我們很早以前就在 Accelerator Model ↗ 詳細寫過。Nvidia 最後真正會 productize 的，是第三代 Groq LPU。

Atomic Claim 60/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0054

Claim: 不過，Groq LPU 2 因設計問題，最終從未產品化。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 61/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0055

Claim: 晶片上的 C2C SerDes 無法達到宣稱的 112G 速度，導致設計無法正常運作。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 62/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0056

Claim: 原文預期：第三代 Groq LPU 將由 Nvidia 推動產品化。
Frame: RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

SRAM and Memory Hierarchy

We have written about the role of SRAM in the memory hierarchy, but the quick recap is that SRAM is very fast (low latency and high bandwidth) but this comes at the expense of density and therefore cost.

我們之前已寫過 SRAM 在 memory hierarchy 的角色；簡單回顧就是：SRAM 非常快，latency 低、bandwidth 高，但代價是 density 低，因此成本很高。

Atomic Claim 63/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0058

Claim: 簡要來說，SRAM 速度非常快，具備低 latency 與高 bandwidth，但代價是較低的 density，因而帶來較高成本。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

SRAM machines such as Groq’s LPU therefore enable very fast time to first token and tokens per second per user but at the expense of total throughput, as their limited SRAM capacity quickly gets saturated by weights, with little left over for KVcache that grows as more users are batched. GPUs win for throughput and cost as we have shown. This is why Nvidia has decided to combine these architectures to get the best of both worlds: accelerate parts of decode that are more latency sensitive and are not as memory heavy on a low-latency SRAM-heavy chip like the LPU, while memory hungry attention is performed on GPUs that come with a lot of fast (but not SRAM fast) memory capacity.

因此 Groq LPU 這類 SRAM machine 可以做到很快的 time-to-first-token 與 tokens/sec/user，但 total throughput 會受限。原因是 SRAM capacity 很快就被 model weight 塞滿，留給 KV cache 的空間很少，而 batch user 越多，KV cache 又會持續增加。正如我們已展示，GPU 在 throughput、cost 上更有優勢。這也解釋 Nvidia 為什麼選擇把兩種 architecture 結合：decode 中對 latency 更敏感、memory footprint 沒那麼大的部分，交給低 latency、SRAM-heavy 的 LPU；memory-hungry attention 則由 GPU 處理，因為 GPU 有大量高速 memory capacity，雖然還是沒有 SRAM 那麼快。

Atomic Claim 64/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0061

Claim: LPU 有限的 SRAM 容量很快就可能被模型權重占滿。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 65/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0060

Claim: Groq 的 LPU 是以總吞吐量換取低延遲。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 66/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0062

Claim: 因此 Nvidia 決定結合兩種架構，以加速 decode 中對延遲較敏感的部分。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 67/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0061::SPLIT01

Claim: 相較 LPU 這類以 SRAM 為核心的機器，GPU 在總 throughput 上更具優勢。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 68/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0063

Claim: The latency-sensitive portions of decode selected for Groq LPU execution are less memory-heavy and suit its low-latency SRAM-heavy architecture.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 69/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0064

Claim: memory-intensive 的 attention 則由 GPU 執行；GPU 擁有大量高速記憶體容量，雖然速度仍不及 SRAM。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 70/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0060::SPLIT03

Claim: 模型 weights 佔用有限的 SRAM 後，LPU 留給 KV cache 的空間很少。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 71/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0060::SPLIT04

Claim: 隨著 batch 中的使用者增加，KV cache 的容量需求會成長。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 72/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0058

Claim: Groq 的 LPU 可實現非常短的 time to first token。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 73/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0061::SPLIT02

Claim: 相較 LPU 這類以 SRAM 為核心的機器，GPU 在成本上更具優勢。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 74/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0059::SPLIT02

Claim: Groq 的 LPU 可提供很高的每位使用者 tokens per second。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 75/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-MEM-HBM4-LATENCY

Claim: In SemiAnalysis’s March 2026 memory comparison, HBM4 12-Hi latency is ~100–150ns.
Frame: ATTRIBUTE · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 76/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-MEM-GDDR7-CAPACITY

Claim: In SemiAnalysis’s March 2026 memory comparison, GDDR7 capacity is ~16–48GB per GPU (~8–12 chips).
Frame: ATTRIBUTE · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 77/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-MEM-HBM4-CAPACITY

Claim: In SemiAnalysis’s March 2026 memory comparison, HBM4 12-Hi capacity is ~288GB per GPU/XPU.
Frame: ATTRIBUTE · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 78/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-MEM-HBM4-BANDWIDTH

Claim: In SemiAnalysis’s March 2026 memory comparison, HBM4 12-Hi bandwidth is ~22TB/s per GPU/XPU.
Frame: ATTRIBUTE · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 79/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-MEM-Groq-LPU-LATENCY

Claim: In SemiAnalysis’s March 2026 memory comparison, LPU SRAM latency is ~5–20ns.
Frame: ATTRIBUTE · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 80/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-MEM-DDR5-BANDWIDTH

Claim: In SemiAnalysis’s March 2026 memory comparison, DDR5 bandwidth is 307–614GB/s per CPU.
Frame: ATTRIBUTE · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 81/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-MEM-Groq-LPU-CAPACITY

Claim: In SemiAnalysis’s March 2026 memory comparison, LPU SRAM capacity is ~500MB per LPU.
Frame: ATTRIBUTE · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 82/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-MEM-GDDR7-LATENCY

Claim: In SemiAnalysis’s March 2026 memory comparison, GDDR7 latency is ~50–80ns.
Frame: ATTRIBUTE · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 83/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-MEM-DDR5-LATENCY

Claim: In SemiAnalysis’s March 2026 memory comparison, DDR5 latency is ~60–100ns.
Frame: ATTRIBUTE · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 84/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-MEM-Groq-LPU-BANDWIDTH

Claim: In SemiAnalysis’s March 2026 memory comparison, LPU SRAM bandwidth is ~150TB/s per LPU.
Frame: ATTRIBUTE · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 85/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-MEM-DDR5-CAPACITY

Claim: In SemiAnalysis’s March 2026 memory comparison, DDR5 capacity is 128–1024GB per CPU (~2–16 DIMMs).
Frame: ATTRIBUTE · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 86/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-MEM-GDDR7-BANDWIDTH

Claim: In SemiAnalysis’s March 2026 memory comparison, GDDR7 bandwidth is ~1.5–1.8TB/s per GPU.
Frame: ATTRIBUTE · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: SemiAnalysis

This brings us to the Groq 3 LPU or LP30, with LPU gen 2 being skipped over. This chip has no Nvidia design involvement. The SerDes issues affecting v2 appear to be fixed. Behind the paywall, we will reveal the SerDes IP vendor which may come as a surprise. Nvidia also announced an LP35 which is a minor refresh of the LP30 which will remain on SF4 and will require a new tapeout. It will incorporate NVFP4 number format but given Nvidia is prioritizing time to market we don’t expect any other drastic design changes.

這就帶到 Groq 3 LPU，也就是 LP30；LPU Gen2 會直接跳過。這顆 chip 沒有 Nvidia design involvement，而影響 v2 的 SerDes issue 看起來已經解決。Paywall 後我們會揭露 SerDes IP vendor，答案可能會讓人意外。Nvidia 也公布 LP35，這是 LP30 的 minor refresh，仍採 SF4，但需要新的 tapeout；它會加入 NVFP4 number format。不過 Nvidia 現在最重視 time-to-market，因此我們不預期其他 design 會有很劇烈的變化。

Atomic Claim 87/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0065::SPLIT02

Claim: Groq 的第二代 LPU 在 roadmap 中被直接跳過。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 88/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0071

Claim: LP35 將支援 NVFP4 數值格式。
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 89/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0069

Claim: LP35 是 LP30 的小幅更新版本。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 90/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0070

Claim: The announced LP35 refresh requires a new tapeout.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 91/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0069::SPLIT01

Claim: Nvidia 宣布了 LP35。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 92/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0066

Claim: LP30 was designed without Nvidia design involvement.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 93/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0067

Claim: SemiAnalysis 判斷，LP30 中影響 LPU v2 的 SerDes 問題看起來已修復。
Frame: ATTRIBUTE · Mode: INFERRED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 94/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0065::SPLIT01

Claim: Groq 第三代 LPU 是 LP30。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 95/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0070

Claim: LP35 將繼續採用 SF4。
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 96/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0072

Claim: SemiAnalysis 不預期 LP35 除支援 NVFP4 外還有其他大幅設計變更，因為 Nvidia 優先追求上市速度。
Frame: CLAIM_ONLY · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 97/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-VXMFUNC

Claim: Nvidia’s Groq 3 LPU slide assigns pointwise math, activations and type conversions to VXM vector operations.
Frame: ATTRIBUTE · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 98/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-LPU-SERDES

Claim: Nvidia’s Groq 3 LPU architecture slide specifies 112Gbps SerDes speed per chip-to-chip connection.
Frame: ATTRIBUTE · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 99/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-MXMFUNC

Claim: Nvidia’s Groq 3 LPU slide assigns dense matrix multiplication to MXM.
Frame: ATTRIBUTE · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 100/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-SXMFUNC

Claim: Nvidia’s Groq 3 LPU slide assigns data movement, permutation, rotation and transposition to SXM.
Frame: ATTRIBUTE · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: Nvidia

LPU 3’s near reticle size die layout is very similar to LPU 1. a significant amount of area taken is up by the 500MB of on-chip SRAM, with a very small amount of area dedicated to MatMul cores that offer 1.2 PFLOPs of FP8 compute – a fraction of compute compared to Nvidia GPUs. This compares to LPU 1 with 230MB of SRAM and 750 TFLOPs of INT8, with the performance increase mostly driven by node migration from GF16 to SF4. As a single monolithic die, advanced packaging isn’t required.

LPU 3 的 near-reticle-size die layout 和 LPU 1 非常相似。Die 上很大一部分面積都被 500MB on-chip SRAM 占用，真正分給 MatMul core 的 area 很小，只提供 1.2 PFLOPs FP8 compute，和 Nvidia GPU 相比只是零頭。LPU 1 則有 230MB SRAM、750 TFLOPs INT8；這次 performance improvement 主要來自製程從 GF16 移到 SF4。因為仍是 single monolithic die，所以不需要 advanced packaging。

Atomic Claim 101/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0075::SPLIT02

Claim: LPU 1 可提供 750 TFLOPs 的 INT8 運算能力。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 102/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0074

Claim: LPU 3 接近 reticle 尺寸的晶粒布局，與 LPU 1 非常相似。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 103/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0074::SPLIT01

Claim: LPU 3 只有很小一部分 die area 配置給 MatMul cores。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 104/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0074::SPLIT03

Claim: LPU 3 的運算能力僅為 Nvidia GPU 的一小部分。
Frame: COMPARISON · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 105/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0074::SPLIT02

Claim: LPU 3 的 MatMul cores 可提供 1.2 PFLOPs 的 FP8 compute。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 106/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0075::SPLIT01

Claim: LPU 1 配備 230MB SRAM。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 107/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0077

Claim: LPU 3 does not require advanced packaging because it is a single monolithic die.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 108/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0075

Claim: A substantial portion of the LPU 3 die area is occupied by its 500MB of on-chip SRAM.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 109/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0076

Claim: The article attributes LPU 3’s performance improvement mainly to migration from “GF16” to SF4; its earlier Gen-1 section instead names GlobalFoundries 14nm.
Frame: CLAIM_ONLY · Mode: ATTRIBUTED · Mapping: PARTIAL
開啟本版 Claim 與證據

One of the benefits of relying on SF4 is that it isn’t constrained like TSMC’s N3, which is putting a cap on accelerator production and is a key reason why the industry remains compute constrained. ↗ This is in addition to not having HBM which is also constrained ↗. This allows Nvidia to ramp production of the LPU without sacrificing or eating into their valuable TSMC allocation or HBM allocations, representing true incremental revenue and capacity that noone else can access.

採 SF4 的一個優點，是不像 TSMC N3 那樣受 capacity constraint。N3 capacity 正在限制 accelerator production，也是整個產業持續 compute constrained 的重要原因 ↗。LPU 同時也不使用另一個供應受限的 HBM ↗。因此 Nvidia 可以 ramp LPU production，而不會犧牲或侵蝕自己珍貴的 TSMC allocation、HBM allocation，形成真正 incremental 的 revenue、capacity，而且這些 capacity 其他 competitor 無法取得。

Atomic Claim 110/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0078

Claim: The article describes Samsung SF4 as less capacity-constrained than TSMC N3, whose constraints limit accelerator output.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 111/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0081::SPLIT02

Claim: 上述不侵蝕 TSMC 與 HBM 配額的 LPU 增產，可為 Nvidia 形成真正的 incremental revenue。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 112/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0079

Claim: TSMC N3 capacity constraints are a key reason the accelerator industry remains compute-constrained in the article’s March 2026 assessment.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 113/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0082

Claim: SemiAnalysis characterizes the LPUs’ use of available SF4 capacity as incremental capacity unavailable to competitors relying on constrained TSMC/HBM allocations.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 114/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0080

Claim: LPU production does not require the HBM supply that is also capacity-constrained.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 115/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0081::SPLIT01

Claim: Nvidia 可以擴大 LPU 產量，而不必犧牲或侵蝕其珍貴的 TSMC 產能配額與 HBM 配額。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Since Nvidia has taken over, the next generation LP40 will be fabricated on TSMC N3P and use CoWoS-R, and Nvidia will contribute more of their own IP such as supporting the NVLink protocol rather than Groq’s C2C. This will be the first LPU to be extremely co-designed alongside the Feynman platform. Groq’s original plans for LPU Gen 4 was also with TSMC and Alchip as the back-end design partner. Alchip’s involvement is now redundant with Nvidia able to perform backend design on their own. One of the technical innovations planned is hybrid bonded DRAM to extend on-chip memory with only a slight decrease in latency and bandwidth vs SRAM, but much higher performance compared to DRAM. SK Hynix was tapped as the supplier of the DRAM to be used for the 3D stacking. All of this and more was detailed long ago in the Accelerator model ↗.

Nvidia 接手後，下一代 LP40 會改用 TSMC N3P、CoWoS-R，而且 Nvidia 會加入更多自家 IP，例如改支援 NVLink protocol，而不是沿用 Groq C2C。這也會是第一代和 Feynman platform 進行 extreme co-design 的 LPU。Groq 原本的 LPU Gen4 計畫同樣是 TSMC 製程，back-end design partner 是 Alchip；現在 Nvidia 自己就能做 backend design，因此 Alchip 的角色已變得多餘。規劃中的一項 technical innovation，是使用 hybrid-bonded DRAM 擴充 on-chip memory；相較 SRAM，latency、bandwidth 只小幅下降，但比一般 DRAM 快很多。SK Hynix 原本被選為 3D stacking DRAM supplier。這些細節我們很早以前就在 Accelerator Model ↗ 說過。

Atomic Claim 116/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0083

Claim: 自 Nvidia 接手後，下一代 LP40 將採用 TSMC N3P 製程。
Frame: CLAIM_ONLY · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 117/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0086

Claim: Nvidia 將在 LP40 中導入更多自有 IP。
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 118/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0085

Claim: LP40 將採用 CoWoS-R。
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 119/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0086

Claim: Groq 原先對 LPU Gen 4 的規劃，同樣是與 TSMC 合作。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 120/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0087

Claim: Under Groq’s original LPU Gen 4 plan, Alchip was the back-end design partner.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 121/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0088

Claim: 由於 Nvidia 已能自行執行 backend design，Alchip 的參與如今變得多餘。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 122/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0085

Claim: LP40 is planned as the first LPU co-designed with Nvidia’s Feynman platform.
Frame: RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 123/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0089

Claim: 規劃中的一項技術創新是採用 hybrid-bonded DRAM 擴充 on-chip memory；相較 SRAM，其延遲與頻寬只略為下降，但相較一般 DRAM 具更高效能。
Frame: COMPARISON · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 124/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0090

Claim: SK Hynix 被選為 3D stacking 所使用 DRAM 的供應商。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 125/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0084::SPLIT03

Claim: Nvidia 將在 LP40 中導入 NVLink protocol，取代 Groq 的 C2C。
Frame: NARY_RELATION · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Source: Nvidia, SemiAnalysis Accelerator Model ↗

GPU and LPU Integration: Attention FFN Disaggregation (AFD)

Source: Nvidia

Now with an understanding of what LPUs are good for we can understand how they fit into inference setups. NVIDIA introduced LPUs to improve the performance of high interactivity scenarios. In those scenarios, LPUs can leverage their low-latency capabilities to improve the decode phase latencies. One way LPUs can improve decode phase latencies is by applying the Attention FFN Disaggregation (AFD) technique, introduced in MegaScale-Infer ↗ and Step-3 ↗.

理解 LPU 的強項後，就能看它如何放進 inference setup。NVIDIA 導入 LPU，主要是為了改善 high-interactivity scenario 的 performance；這類情境可以利用 LPU low-latency 特性，降低 decode phase latency。其中一種方式，就是使用 MegaScale-Infer ↗、Step-3 ↗ 提出的 Attention-FFN Disaggregation（AFD）。

Atomic Claim 126/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0095::SPLIT02

Claim: AFD 技術曾在 MegaScale-Infer 中提出。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 127/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0093

Claim: NVIDIA 導入 LPU，以提升高互動情境下的效能。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 128/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0095::SPLIT01

Claim: LPU 可透過 Attention FFN Disaggregation（AFD）改善 decode 階段延遲。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 129/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0095::SPLIT03

Claim: AFD 技術也曾在 Step-3 中提出。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 130/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0094

Claim: 在這類情境下，LPU 可利用低延遲能力，改善 decode 階段的延遲。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

As we explained in our InferenceX article ↗, LLM inference involves two phases: prefill and decode. Prefill processes the full input context: It is compute-intensive, which is suitable for GPUs. On the other hand, decode predicts new tokens and is memory-bounded. Decode is latency-sensitive because the model predicts new tokens one by one, and LPU’s high SRAM bandwidth and low-latency capabilities can help accelerate this iterative process.

如我們 InferenceX 文章 ↗ 所解釋，LLM inference 有兩個 phase：prefill、decode。Prefill 會處理完整 input context，compute intensity 高，因此適合 GPU；decode 則是一個 token 一個 token 預測，屬 memory-bound、又對 latency 很敏感。LPU 的 high SRAM bandwidth、low latency，正好可以幫助加速這個 iterative decode process。

Atomic Claim 131/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0100

Claim: Groq LPU high SRAM bandwidth and low latency can accelerate the iterative token-by-token decode process.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 132/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0099

Claim: Decode 對延遲非常敏感，因為模型必須逐一預測新的 token。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 133/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0097

Claim: Prefill 會處理完整 input context；它屬於 compute-intensive 工作，因此適合由 GPU 執行。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 134/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0098

Claim: 另一方面，decode 會預測新的 token，且屬於 memory-bounded 工作。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 135/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0096

Claim: 如 InferenceX 文章所述，LLM inference 包含兩個階段：prefill 與 decode。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: SemiAnalysis

Attention and FFN are subsets of operations in a model. In a model forward pass, attention’s output feeds into a token router, and the token router assigns each token to k experts, where each expert is an FFN. Attention and FFN have very different performance properties. During decode phase, the GPU utilization of attention barely improves when scaling batch size due to being bounded by loading KV cache. In contrast, the GPU utilization of FFN scales with batch size comparatively better.

Attention、FFN 都是 model 中的 operation subset。一次 forward pass 裡，attention output 會送進 token router，由 router 把每個 token 指派給 top-k expert，而每個 expert 本質上就是一個 FFN。Attention、FFN 的 performance property 很不一樣：decode phase 裡，attention 因為受 KV cache loading 限制，就算增加 batch size，GPU utilization 也幾乎不會改善；FFN 則相對會隨 batch size 增加而更有效率。

Atomic Claim 136/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0104

Claim: 在 decode 階段，attention 的 GPU utilization 在提高 batch size 時幾乎不會改善，因為受到載入 KV cache 的限制。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 137/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0102

Claim: 關於 AFD：在模型 forward pass 中，attention 的輸出會送入 token router。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 138/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0103::SPLIT01

Claim: token router 會將每個 token 分配給 k 個 experts。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 139/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0103::SPLIT02

Claim: token router 所分配的每個 expert 都是一個 FFN。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 140/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0105

Claim: Attention 與 FFN 具有非常不同的效能特性。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 141/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0101

Claim: Attention 與 FFN 都是模型中的 operation subset。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 142/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0105

Claim: 相較之下，GPU 執行 FFN 時的 utilization，隨 batch size 增加而改善的幅度相對更好。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

This is something we have worked with certain hardware vendors and memory companies on with our inference simulator for more than 6 months. ↗

這也是過去超過六個月，我們用 inference simulator 和部分 hardware vendor、memory company 一直共同研究的主題 ↗。

Source: MegaScale-Infer, SemiAnalysis

As state-of-the-art mixture-of-expert (MoE) models grow increasingly sparse, tokens can choose experts from a larger expert pool. As a result, each expert receives fewer tokens, leading to lower utilization. This motivates attention and FFN disaggregation. If a GPU only performs attention operations, its HBM capacity can be fully allocated to KV cache, increasing the total number of tokens it can process, which then increases the tokens each expert processes on average.

隨 SOTA MoE model 越來越 sparse，token 可以從更大的 expert pool 中選擇，結果每個 expert 平均收到的 token 變少，utilization 下降。這就是 Attention、FFN disaggregation 的動機。如果某顆 GPU 只負責 attention，整個 HBM capacity 都能分給 KV cache，提高它可同時處理的 token 數量；token 總量增加後，每個 expert 平均能分到的 token 也會增加。

Atomic Claim 143/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0110

Claim: 如果 GPU 只執行 attention operation，其 HBM 容量就能全部分配給 KV cache，提高可處理的 token 總數，進而增加每個 expert 平均處理的 token 數。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 144/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0107

Claim: 隨著 state-of-the-art Mixture-of-Experts 模型變得愈來愈 sparse，token 可以從更大的 expert pool 中選擇 expert。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 145/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0108

Claim: In the described sparse-MoE scaling scenario, an expanded expert pool gives each expert fewer tokens, reducing utilization.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Source: SemiAnalysis

Comparing the two operations, we see attention is stateful due to dynamic KV cache loading patterns, whereas FFN is stateless since the computation only depends on the token inputs. Thus, we disaggregate the computation of attention and FFN. We map attention computations to GPUs, which handle dynamic workloads well. For FFNs, we map them to LPUs, since LPU architecture is inherently deterministic and benefits from static compute workloads.

比較兩種 operation，attention 是 stateful，因為 KV cache loading pattern 會動態變化；FFN 則是 stateless，computation 只取決於 token input。因此可以把 attention、FFN computation 拆開：attention map 到善於處理 dynamic workload 的 GPU；FFN 則 map 到 LPU，因為 LPU architecture 本質上 deterministic，更適合 static compute workload。

Atomic Claim 146/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0114

Claim: attention computation 會被分配到 GPU，因為 GPU 擅長處理 dynamic workload。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 147/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0113

Claim: The described disaggregated-decode design separates attention and FFN computation.
Frame: CLAIM_ONLY · Mode: INFERRED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 148/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0111

Claim: 比較兩種運算後，SemiAnalysis 認為 attention 因為有動態 KV cache 載入模式，因此屬於 stateful 工作負載。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 149/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0112

Claim: FFN 是 stateless，因為其運算只依賴 token input。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 150/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0115

Claim: FFN 會被分配到 LPU，因為 LPU 架構本質上是 deterministic，並能受益於 static compute workload。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: SemiAnalysis, MegaScale-Infer

With AFD, token routing from GPUs to LPUs can become the bottleneck, especially under strict latency constraints. The token routing flow involves two operations: dispatch and combine. In the dispatch step, we route each token to their top k experts with an All-to-All collective operation. After experts complete their computation, we perform the combine step, where the outputs are sent back to the source location with a reverse All-to-All collective, continuing the next layer’s computation.

AFD 下，GPU 到 LPU 的 token routing 可能變成 bottleneck，尤其 latency constraint 很嚴時。Token routing 有兩個 operation：dispatch、combine。Dispatch 階段用 All-to-All collective，把每個 token route 到 top-k expert；expert 計算完成後，combine 再用 reverse All-to-All，把 output 傳回 source location，接著繼續下一層 computation。

Atomic Claim 151/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0116

Claim: 在 AFD 下，從 GPU 到 LPU 的 token routing 可能成為 bottleneck，尤其是在嚴格 latency constraint 下。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 152/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0118

Claim: 在 dispatch 階段，每個 token 會透過 All-to-All collective operation 被送到其 top-k experts。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 153/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0119

Claim: expert 完成運算後會進入 combine 階段，輸出透過 reverse All-to-All collective 傳回來源位置，接著繼續下一層運算。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 154/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0117

Claim: token routing 流程包含兩個 operation：dispatch 與 combine。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Source: SemiAnalysis

To hide the communication latency of dispatch and combine, we employ ping pong pipeline parallelism. In addition to splitting batches into micro-batches and computation pipelining like standard pipeline parallelism, the tokens dispatched to the LPUs are combined back to the source GPUs, so they ping pong between the GPUs and the LPUs.

為了隱藏 dispatch、combine 的 communication latency，可以使用 ping-pong pipeline parallelism。除了像一般 pipeline parallelism 一樣，把 batch 切成 micro-batch 並做 computation pipelining，送到 LPU 的 token 計算完後還要 combine 回 source GPU，因此 token 會在 GPU、LPU 之間來回 ping-pong。

Atomic Claim 155/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0120

Claim: 為了隱藏 dispatch 與 combine 的 communication latency，系統採用 ping-pong pipeline parallelism。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 156/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0121::SPLIT02

Claim: 送到 LPU 的 token 會被 combine 回來源 GPU，因此 token 會在 GPU 與 LPU 之間來回 ping-pong。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 157/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0121::SPLIT01

Claim: 標準 pipeline parallelism 會將 batch 拆成 micro-batches 並進行 computation pipelining。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: MegaScale-Infer

Source: SemiAnalysis

Speculative Decoding

A different way LPUs could improve decode phase latencies is by accelerating a speculative decoding setup, where we deploy draft models or Multi-Token Prediction (MTP) layers onto LPUs.

另一種利用 LPU 降低 decode phase latency 的方式，是加速 speculative decoding：把 draft model 或 Multi-Token Prediction（MTP）layer 部署在 LPU 上。

Atomic Claim 158/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0122

Claim: LPU 改善 decode 階段延遲的另一種方式，是加速 speculative decoding setup，把 draft model 或 Multi-Token Prediction（MTP） layer 部署到 LPU 上。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟本版 Claim 與證據

For a decoding step of context N tokens, adding k additional tokens during forward pass (a warm prefill of k new tokens) marginally increases the latency when k << N. Using this property, speculative decoding uses a small draft model or MTP layers to predict k new tokens, saving time since small models have lower latency per decode step. To verify the draft tokens, the main model only needs one warm prefill of k new tokens, at the latency cost of roughly a single decode step. Speculative decoding usually boosts output token per decode step by 1.5 to 2 tokens, depending on the draft model / MTP accuracy. With its low latency capabilities, LPUs can further increase the latency savings and improve throughput.

對 context 已有 N token 的 decode step 而言，只要 k << N，在 forward pass 額外加入 k 個 token，也就是做一次 k-token warm prefill，latency 增量其實很小。Speculative decoding 就利用這個特性：先讓小型 draft model 或 MTP layer 預測 k 個新 token；因 small model 每 decode step latency 更低，所以能節省時間。驗證這些 draft token 時，main model 只需要做一次 k-token warm prefill，latency cost 大致接近單一步 decode。依 draft model／MTP accuracy 不同，speculative decoding 通常可以把每個 decode step 的 output token 數提高到約 1.5–2 個。LPU latency 更低，因此還能進一步放大這些 latency saving、提高 throughput。

Atomic Claim 159/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0124

Claim: 利用上述特性，speculative decoding 會使用較小的 draft model 或 MTP layer 預測 k 個新 token；因小模型每個 decode step 的 latency 較低，因此可以節省時間。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 160/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0126

Claim: Speculative decoding 通常能把每個 decode step 的 output token 提升到 1.5～2 個，實際幅度取決於 draft model／MTP 的準確率。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 161/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0127::SPLIT02

Claim: 憑藉低 latency 能力，LPU 可改善 throughput。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 162/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0125

Claim: 為了驗證 draft token，main model 只需要對 k 個新 token 做一次 warm prefill，其 latency cost 約等於一次 decode step。
Frame: COMPARISON · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 163/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0123

Claim: 對 context 為 N 個 token 的 decoding step，在 forward pass 中額外加入 k 個 token（即對 k 個新 token 做 warm prefill）時，只要 k 遠小於 N，增加的 latency 就很有限。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 164/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0127::SPLIT01

Claim: 憑藉低 latency 能力，LPU 可進一步提高 latency savings。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: SemiAnalysis

For LPUs, deploying a draft model or MTP layers is quite different from applying AFD. FFNs are stateless, while draft models and MTP layers require dynamic KV cache loading. Each FFN is around hundreds of megabytes, whereas draft models and MTP layers take up tens of gigabytes. To support this memory usage, LPUs can access up to 256 GB of DDR5 per Fabric Expansion Logic FPGAs on the LPX compute tray.

對 LPU 來說，部署 draft model／MTP layer 和做 AFD 很不一樣。FFN 是 stateless；draft model、MTP layer 則需要動態載入 KV cache。單個 FFN 通常只有數百 MB，但 draft model、MTP layer 會吃到數十 GB。為支援這些 memory usage，LPX compute tray 上的每顆 Fabric Expansion Logic FPGA 都可 access 最多 256GB DDR5。

Atomic Claim 165/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0132

Claim: In the source’s speculative-decode discussion, draft models and MTP layers occupy tens of GB, exceeding one LPU’s SRAM capacity.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 166/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0131

Claim: 每個 FFN 的容量約為數百 MB。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 167/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0133

Claim: 為了支援這種 memory usage，LPU 可透過 LPX compute tray 上每顆 Fabric Expansion Logic FPGA 存取最高 256GB 的 DDR5。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 168/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0131

Claim: FFN 屬於 stateless 工作負載。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 169/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0129::RESEG01

Claim: draft model 需要動態載入 KV cache。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 170/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0130

Claim: MTP layer 需要動態載入 KV cache。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

LPX Rack System

Let’s look at the LPX rack system, which has interesting details. Nvidia has displayed an LPX rack with 32 1U LPU compute trays with 2 Spectrum-X switches. This 32 tray 1U version that Nvidia has shown off at GTC is very close to Groq’s original server design before the acquisition. We believe that this server configuration is not the version that will be shipped in 3Q, with Nvidia implementing changes. Here, we will detail what we know about the actual production version. This was already detailed in the Accelerator model ↗.

接著看 LPX rack system，裡面有一些很有意思的細節。Nvidia 在 GTC 展示的是一個 32 個 1U LPU compute tray + 2 台 Spectrum-X switch 的 LPX rack。這個 32-tray、1U 版本其實非常接近 Groq 被收購前原本的 server design。我們認為這不是第三季真正會出貨的 production version，Nvidia 還會做修改。下面會整理我們目前知道的實際 production version；這些資訊先前已在 Accelerator Model ↗ 提過。

Atomic Claim 171/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0135::SPLIT01

Claim: Nvidia 展示過一套 LPX rack，內含 32 個 1U LPU compute tray。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 172/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0136

Claim: Nvidia 在 GTC 展示的這個 32-tray、1U 版本，與 Groq 在收購前的原始 server design 非常接近。
Frame: COMPARISON · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 173/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0135::SPLIT02

Claim: Nvidia 展示過的 LPX rack 內含 2 個 Spectrum-X switch。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 174/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0137

Claim: SemiAnalysis 認為這個 server configuration 並不是 3Q 會出貨的版本，Nvidia 仍會進行修改。
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 175/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-LPX-POWER

Claim: The LPX rack layout shows four 90kW power shelves.
Frame: ATTRIBUTE · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 176/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-LPX-OOB

Claim: The LPX rack layout includes one out-of-band 1GbE management switch.
Frame: ATTRIBUTE · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: SemiAnalysis Accelerator Model ↗

LPX Compute Tray

Each LPX compute tray or node has 16 LPUs with 2 Altera FPGAs, 1 Intel Granite Rapids host CPU and 1 BlueField-4 front-end module. As with other Nvidia systems, hyperscalers customers can and will use their own Front-end NIC of choice rather than paying for Nvidia’s BlueField.

每個 LPX compute tray／node 內含 16 顆 LPU、2 顆 Altera FPGA、1 顆 Intel Granite Rapids host CPU，以及 1 個 BlueField-4 frontend module。和其他 Nvidia system 一樣，hyperscaler customer 可以、也會使用自己選擇的 frontend NIC，而不是額外付費買 Nvidia BlueField。

Atomic Claim 177/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0141::RESEG01

Claim: 和其他 Nvidia 系統相同，hyperscaler 客戶可以、也會使用自己選擇的 Front-end NIC，而不是支付 Nvidia 的 BlueField。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 178/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0140::SPLIT02

Claim: 每個 LPX compute tray／node 配置 2 顆 Altera FPGA。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 179/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0142

Claim: 每個 LPX compute tray／node 配備 1 個 BlueField-4 front-end module。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 180/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0140::SPLIT01

Claim: 每個 LPX compute tray／node 配置 16 顆 LPU。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 181/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0141

Claim: 每個 LPX compute tray／node 配備 1 顆 Intel Granite Rapids host CPU。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: SemiAnalysis Accelerator Model ↗

The LPU modules are mounted in a belly-to-belly on the PCB, meaning 8 LP30 modules on the top side of the PCB and the other 8 LP30 modules on the bottom. All of the connectivity that comes out of the LPU are via PCB traces and given the dense all-to-all mesh for intra-node connections this requires a very high spec PCB to support the routing. The belly-to-belly mounting is used to reduce PCB trace lengths across the ‘X’ and ‘Y’ dimensions.

LPU module 採 belly-to-belly 方式裝在 PCB：8 個 LP30 module 在 PCB 正面，另外 8 個在背面。LPU 所有 connectivity 都走 PCB trace；又因 intra-node 是非常密集的 all-to-all mesh，routing 對 PCB specification 要求非常高。Belly-to-belly mounting 的目的，就是縮短 X、Y 維度上的 PCB trace length。

Atomic Claim 182/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0144

Claim: LPU 對外的所有 connectivity 都透過 PCB trace。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 183/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0146

Claim: belly-to-belly mounting 用來縮短 PCB 在 X 與 Y 維度上的 trace length。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 184/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0143

Claim: LPU module 在 PCB 上採 belly-to-belly 安裝：8 個 LP30 module 位於 PCB 上側，另外 8 個位於下側。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 185/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0145

Claim: 由於 intra-node connection 採 dense all-to-all mesh，需要非常高規格的 PCB 才能支援 routing。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Source: SemiAnalysis Networking Model ↗

Something interesting about the system is the important role the FPGAs play. Nvidia refers to the FPGAs as “Fabric Expansion Logic” which serves multiple purposes. First, they act as a NIC which converts the LPU’s C2C protocol into Ethernet to connect to the Spectrum-X based ethernet scale-out fabric. It is this scale-out fabric through which the LPUs connect to GPUs in the decode system.

這套 system 很有意思的一點，是 FPGA 扮演非常重要的角色。Nvidia 把 FPGA 稱為『Fabric Expansion Logic』，用途很多。第一，它本身扮演 NIC，把 LPU 的 C2C protocol 轉成 Ethernet，接到 Spectrum-X-based Ethernet scale-out fabric。LPU 就是透過這套 scale-out fabric，連到 decode system 裡的 GPU。

Atomic Claim 186/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0149

Claim: 首先，這些 FPGA 會充當 NIC，把 LPU 的 C2C protocol 轉成 Ethernet，以連接到以 Spectrum-X 為基礎的 Ethernet scale-out fabric。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 187/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0150

Claim: LPU 就是透過這個 scale-out fabric，在 decode system 中連接到 GPU。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 188/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0148::SPLIT01

Claim: Nvidia refers to the LPX compute-tray Altera FPGAs as Fabric Expansion Logic.
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Second, the LPUs also traverse through the FPGAs to reach the host CPU, with the FPGAs converting C2C to PCIe to the CPU.

第二，LPU 要連 host CPU 也必須經過 FPGA，由 FPGA 把 C2C 轉成 PCIe，再連到 CPU。

Atomic Claim 189/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0151

Claim: 其次，LPU 也會經過 FPGA 才能連到 host CPU；FPGA 會把 C2C 轉成 PCIe 再連至 CPU。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Third, the FPGAs are connected to the backplane to talk to other FPGAs in the node, we believe this is to help manage control flow and timing of all the LPUs. The FPGAs also bring extra system DRAM of up to 256GB each. This pool of memory can be used for KVCache if the user wants the entire decode process served by the LPX.

第三，FPGA 也接到 backplane，和 node 內其他 FPGA 溝通；我們認為這主要用來協調所有 LPU 的 control flow、timing。每顆 FPGA 另外可帶最多 256GB system DRAM；如果使用者希望整個 decode process 都在 LPX 上完成，這池 memory 也可以拿來存 KV cache。

Atomic Claim 190/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0153

Claim: 每顆 FPGA 還可提供最高 256GB 的額外 system DRAM。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 191/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0154

Claim: 如果使用者希望整個 decode process 都由 LPX 提供服務，這個 memory pool 可以拿來存放 KVCache。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 192/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0152::SPLIT01

Claim: FPGA 會連到 backplane，以和 node 內其他 FPGA 通訊。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 193/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0152::SPLIT02

Claim: SemiAnalysis 認為 FPGA 與 backplane 的這種連線，可能用來協助管理所有 LPU 的 control flow 與 timing。
Frame: NARY_RELATION · Mode: INFERRED · Mapping: COMPLETE
開啟本版 Claim 與證據

On the front panel there are 8 x OSFP cages for cross-rack C2C, while there will be 2 cages (likely QSFP-DD) that goes to the Spectrum-switches that is used to connect the LPUs and the GPUs for the disaggregated decode system. We will share more about this when we describe the network.

Front panel 有 8 個 OSFP cage，供 cross-rack C2C 使用；另外還有 2 個 cage——很可能是 QSFP-DD——連到 Spectrum switch，用來把 LPU、GPU 接進 disaggregated decode system。等後面講 network 時會再展開。

Atomic Claim 194/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0157::RESEG01

Claim: The production LPX compute-tray connection to Spectrum switches links LPUs to GPUs for disaggregated decode.
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 195/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0155

Claim: Each production LPX compute-tray front panel is described as having eight OSFP cages for cross-rack C2C.
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 196/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0156::RESEG01

Claim: The LPX compute-tray front panel is described as having two cages connected to Spectrum switches.
Frame: CLAIM_ONLY · Mode: EXPECTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 197/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0156::RESEG02

Claim: SemiAnalysis expects the LPX compute-tray’s two Spectrum-facing cages to use QSFP-DD.
Frame: CLAIM_ONLY · Mode: EXPECTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

LPU Network

The LPU network can be divided into the scale-up ‘C2C’ network and scale-out network which interacts with the Nvidia GPUs through Spectrum-X. First let’s discuss the scale-up network which can be divided into 3 portions: intra-node, inter-node/intra-rack, inter-rack. For C2C within the rack Nvidia announced a total of 640TB/s of scale up bandwidth per rack which comes from 256 LPUs x 90 lanes x 112Gbps/8 x 2 directions = 645TB/s. Note that Nvidia uses the total 112G line rate rather than 100G of effective data rate.

LPU network 可以分成 scale-up『C2C』network，以及透過 Spectrum-X 和 Nvidia GPU 互動的 scale-out network。先看 scale-up，它又可分三段：intra-node、inter-node/intra-rack、inter-rack。Rack 內 C2C 部分，Nvidia 宣布每 rack scale-up bandwidth 總計 640TB/s，計算來自 256 LPUs × 90 lanes × 112Gbps ÷ 8 × 2 directions = 645TB/s。要注意，Nvidia 用的是完整 112G line rate，而不是約 100G effective data rate。

Atomic Claim 198/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0162::SPLIT02

Claim: 依 256 顆 LPU × 90 lanes × 112Gbps ÷ 8 × 2 directions 計算，rack 內 C2C 的 scale-up bandwidth 約為 645TB/s。
Frame: ATTRIBUTE · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 199/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0161

Claim: scale-up network 可以分成 3 個部分：intra-node、inter-node／intra-rack、以及 inter-rack。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 200/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0159

Claim: The LPU network includes a scale-up chip-to-chip (C2C) network.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 201/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0162::SPLIT01

Claim: Nvidia 宣布，rack 內 C2C 的每櫃總 scale-up bandwidth 為 640TB/s。
Frame: ATTRIBUTE · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 202/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0160

Claim: scale-out network 會透過 Spectrum-X 與 Nvidia GPU 互動。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 203/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0163

Claim: 要注意的是，Nvidia 採用總 112G line rate，而不是 100G effective data rate。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Intra-Tray Topology

Source: SemiAnalysis Networking Model ↗

Within each tray or node, all 16 LPUs are connected to each other in an all-to-all mesh. Each LPU module connects to the 15 other LPUs within the node with 4x100G of C2C bandwidth. Note that this ‘C2C’ is not related to NVLink, but Groq’s own scaleup fabric. These connections are all via PCB trace, which necessitates an extremely high spec PCB to support this routing density. This is why the belly-to-belly layout is used: it reduces the ‘X’ and ‘Y’ distance between all the LPUs and instead have routing go in the ‘Z’ dimension.

每個 tray／node 內，16 顆 LPU 全部做 all-to-all mesh。每顆 LPU 用 4×100G C2C bandwidth 連到同 node 另外 15 顆 LPU。這裡的『C2C』和 NVLink 無關，是 Groq 自己的 scale-up fabric。所有 connection 都走 PCB trace，因此需要極高規格 PCB 才能承受 routing density。這也是採 belly-to-belly layout 的原因：縮短 LPU 間 X、Y 距離，讓 routing 更多改走 Z dimension。

Atomic Claim 204/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0169::RESEG01

Claim: The LPX belly-to-belly placement routes signals through the Z dimension instead.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 205/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0164

Claim: 每個 tray／node 內的 16 顆 LPU 彼此以 all-to-all mesh 連接。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 206/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0167

Claim: 這些連線全部走 PCB trace，因此需要極高規格的 PCB 才能支撐這種 routing density。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 207/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0165

Claim: 每個 LPU module 會以 4×100G 的 C2C bandwidth，連接 node 內另外 15 顆 LPU。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 208/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0166

Claim: 這裡的 C2C 與 NVLink 無關，而是 Groq 自有的 scale-up fabric。
Frame: COMPARISON · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 209/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0168::RESEG01

Claim: The LPX belly-to-belly LPU placement reduces distances between LPUs in the X and Y dimensions.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

The LPU also has 1x100G going to one FPGA, with each FPGA interfacing with 8 LPUs. The 2 FPGAs each have 8x PCIe Gen 5 going to the CPUs. The LPU needs to traverse through the FPGA to interface with the CPU as LPUs don’t have PCIe PHYs to interface directly.

每顆 LPU 另外有 1×100G 連到一顆 FPGA，每顆 FPGA 負責 8 顆 LPU。兩顆 FPGA 各自再用 8×PCIe Gen5 連到 CPU。因為 LPU 本身沒有 PCIe PHY，無法直接和 CPU interface，所以一定得透過 FPGA。

Atomic Claim 210/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0172

Claim: LPU 必須經過 FPGA 才能和 CPU 介接，因為 LPU 沒有 PCIe PHY 可以直接連線。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 211/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0171

Claim: 2 顆 FPGA 各自都有 8× PCIe Gen 5 連到 CPU。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 212/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0170

Claim: LPU 另有 1×100G 連到一顆 FPGA，而每顆 FPGA 會介接 8 顆 LPU。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Inter-node/Intra-rack

Source: SemiAnalysis Networking Model ↗

Each LPU connects to one LPU from each of the 15 other nodes in the server. Each of these inter-node links is 2x100G so there are 15x2x100G inter-node links coming out of each LPU. These inter-node links are via a copper cable backplane. In addition, each FPGA also connects to an FPGA in every other node at either 25G or 50G per link for 15x25G/50G. This also goes through the backplane. This means that each node has 16 x 15 x 2 lanes for inter-node C2C and 2 x 15 lanes for inter-node FPGA which is a total of 510 lanes or 1020 differential pairs (for Rx and Tx). Therefore, the backplane is 16 x 1020/2 = 8,160 differential pairs – we divide by 2 as each device Tx channel is a corresponding device’s Rx channel.

每顆 LPU 還會各自連到 server 其餘 15 個 node 中的一顆 LPU。每條 inter-node link 是 2×100G，因此每顆 LPU 有 15×2×100G inter-node link，走 copper cable backplane。除此之外，每顆 FPGA 也會連到其餘每個 node 的一顆 FPGA，每 link 可能是 25G 或 50G，因此總共 15×25G/50G，同樣走 backplane。換算下來，每 node 有 16×15×2 lanes 的 inter-node C2C，再加 2×15 lanes 的 inter-node FPGA，合計 510 lanes，若 Rx、Tx 分開就是 1,020 differential pairs。整個 backplane 因此有 16×1,020÷2 = 8,160 differential pairs；除以 2 是因為一個 device 的 Tx channel 對應另一個 device 的 Rx channel。

Atomic Claim 213/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0175

Claim: 這些 inter-node links 透過 copper cable backplane 傳輸。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 214/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0173

Claim: 每顆 LPU 都會連到 server 其餘 15 個 node 中各一顆 LPU。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 215/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0177

Claim: LPX inter-node FPGA links traverse the copper backplane.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 216/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0179

Claim: In the LPX compute-tray inter-node budget, the FPGA links contribute 2×15 lanes, bringing the C2C-plus-FPGA total to 510 lanes or 1,020 Rx/Tx differential pairs.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 217/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0176

Claim: 此外，每顆 FPGA 都會連到其他每個 node 中的一顆 FPGA，每條 link 為 25G 或 50G，合計為 15×25G／50G。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 218/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0180

Claim: 因此 backplane 共有 16×1020÷2＝8,160 differential pairs；之所以除以 2，是因為每個 device 的 Tx channel 對應另一個 device 的 Rx channel。
Frame: ATTRIBUTE · Mode: INFERRED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 219/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0174

Claim: 每條 inter-node link 為 2×100G，因此每顆 LPU 對外有 15×2×100G 的 inter-node links。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 220/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0178

Claim: 因此，每個 node 都有 16×15×2 lanes 用於 inter-node C2C。
Frame: ATTRIBUTE · Mode: INFERRED · Mapping: COMPLETE
開啟本版 Claim 與證據

Inter-rack

Source: SemiAnalysis Networking Model ↗

Lastly, there is the inter-rack C2C. Each LPU has 4x100G lanes that go to the OSFP cages to connect LPUs across 4 racks. There are various configurations that can be used for this inter-rack scale up. One option is 4x100G from each LPU going to one OSFP cage, each OSFP escaping 800G of C2C from 2 LPUs. However, for greater fan out the preferred configuration seems to be each 100G lane from the LPU going to 4 individual cages, with each cage escaping 800G of C2C from 8 LPUs. In terms of how the racks are networked together it appears to be a daisy chain configuration, with each Node0 connected to 2 other Node 0. This can all be achieved within the reach of 100G AECs, though optics can be used if necessary.

最後是 inter-rack C2C。每顆 LPU 有 4×100G lane 走到 OSFP cage，用來跨最多 4 個 rack 連接 LPU。Inter-rack scale-up 有多種 configuration。一種做法是每顆 LPU 的 4×100G 全部進同一個 OSFP cage，每個 OSFP 匯出來自 2 顆 LPU、合計 800G C2C。但若追求更大 fan-out，目前比較偏好的配置，是把每顆 LPU 的四條 100G lane 分別送到四個獨立 cage，每個 cage 匯出 8 顆 LPU 各一條 100G、合計 800G C2C。Rack-to-rack topology 看起來會採 daisy chain，例如每個 Node0 連到另外兩個 Node0。這些距離基本上都能用 100G AEC 覆蓋，必要時也可用 optics。

Atomic Claim 221/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0185

Claim: 若要更大的 fan-out，偏好的 configuration 看起來是讓 LPU 的每條 100G lane 分別連到 4 個 cage；每個 cage 會從 8 顆 LPU 對外導出 800G 的 C2C。
Frame: NARY_RELATION · Mode: INFERRED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 222/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0186

Claim: 關於 LPX：機櫃之間看起來採用 daisy-chain 方式串接 together。
Frame: ATTRIBUTE · Mode: INFERRED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 223/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0188

Claim: The described LPX inter-rack C2C links fit within 100G AEC reach, with optics available if needed.
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 224/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0182

Claim: 每顆 LPU 有 4×100G lanes 連到 OSFP cage，用來跨 4 個 rack 連接 LPU。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 225/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0183

Claim: The LPX inter-rack C2C design supports multiple alternative configurations.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 226/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0187

Claim: 在 SemiAnalysis 判斷可能採用的 LPX inter-rack daisy-chain 配置中，每個 Node0 連接另外兩個 Node0。
Frame: CLAIM_ONLY · Mode: INFERRED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 227/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0181

Claim: The Groq LPU scale-up network includes inter-rack C2C connectivity.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 228/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0184

Claim: 其中一種 configuration，是每顆 LPU 的 4×100G 連到一個 OSFP cage；每個 OSFP 會從 2 顆 LPU 對外導出 800G 的 C2C。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Nvidia’s CPO Roadmap

NVIDIA revealed its CPO Roadmap at the GTC Keynote 2026, with Jensen following up with additional commentary in the Financial Analyst Q+A meeting held the following day. Though many had their hopes up for CPO to be used for scale-up within the rack for Rubin Ultra Kyber, Nvidia’s focus was instead on using CPO to enable larger world size compute systems.

NVIDIA 在 GTC 2026 Keynote 公布 CPO roadmap，Jensen 隔天在 Financial Analyst Q+A 又補充更多說明。很多人原本期待 Rubin Ultra Kyber rack 內的 scale-up 就會使用 CPO，但 Nvidia 的重點其實是用 CPO 建立更大的 world-size compute system。

Atomic Claim 229/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0189::SPLIT02

Claim: Jensen Huang 在 GTC Keynote 2026 隔天舉行的 Financial Analyst Q+A meeting 中，對 CPO roadmap 補充評論。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 230/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0190::SPLIT02

Claim: Nvidia 的重點反而是使用 CPO 來支援更大的 world-size compute systems。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 231/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0190::SPLIT01

Claim: 許多人原先期待 CPO 會用於 Rubin Ultra Kyber rack 內的 scale-up。
Frame: NARY_RELATION · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 232/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0189::SPLIT01

Claim: NVIDIA 在 2026 年 GTC Keynote 公布 CPO roadmap。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: SemiAnalysis AI Networking Model ↗, Nvidia

In the Rubin Generation, Nvidia will offer the Rubin GPU in an Oberon NVL72 form factor with an all-copper scale-up network. For Rubin Ultra, as we expected, there will only be a copper scale-up option for Rubin Ultra in the Oberon and Kyber Rack form factor. Rubin Ultra will also be offered in a larger world size system that connects 8 Oberon Racks of 72 Rubin Ultra GPUs to form what will be referred to as NVL576. CPO scale-up will be used to build the larger world size, connecting between the racks in a two-tier all to all network, though scale-up inside the racks will remain copper-based.

Rubin 世代，Nvidia 會以 Oberon NVL72 form factor 提供 Rubin GPU，rack 內 scale-up 全部使用 copper。Rubin Ultra 方面，如我們原先預期，Oberon、Kyber rack form factor 的 rack 內 scale-up 也只有 copper option。不過 Rubin Ultra 還會提供更大的 world-size system：把 8 個 Oberon rack、每 rack 72 顆 Rubin Ultra GPU 串起來，形成 NVL576。這種較大 world size 會使用 CPO scale-up，在 rack 之間用 two-tier all-to-all network 互連；但 rack 內 scale-up 仍維持 copper。

Atomic Claim 233/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0194

Claim: 較大的 world-size 系統會用 CPO scale-up 做 rack 間連接，採 two-tier all-to-all network；rack 內的 scale-up 則仍維持 copper。
Frame: COMPARISON · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 234/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0193

Claim: Rubin Ultra 也會提供較大 world-size 的系統：將 8 個 Oberon Rack、每櫃 72 顆 Rubin Ultra GPU 連在一起，形成 NVL576。
Frame: NARY_RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 235/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0192

Claim: 對 Rubin Ultra 而言，在 Oberon 與 Kyber Rack form factor 中只會提供 copper scale-up 選項。
Frame: NARY_RELATION · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 236/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0191

Claim: 在 Rubin generation，Nvidia 將以 Oberon NVL72 form factor 提供 Rubin GPU，並採全 copper 的 scale-up network。
Frame: NARY_RELATION · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

When we reach the Feynman Generation, CPO usage will expand via another large world size rack, the NVL1152 which is formed by combining 8 Kyber racks. While the Nvidia Technical Blog ↗ that outlines the rack configuration roadmap states that “NVIDIA Kyber will scale up into a massive all-to-all NVL1152 supercomputer using similar direct optical interconnects for rack-to-rack scale-up”, Jensen Huang in a Financial Analyst Q+A session did say that NVL1152 in Feynman would be “all CPO”. There is some disagreement on whether copper will still be used for scale-up within the rack or whether CPO will replace copper.

到了 Feynman 世代，CPO 使用範圍會進一步擴張，推出另一套更大 world-size system——NVL1152，由 8 個 Kyber rack 組成。Nvidia Technical Blog ↗ 的 rack roadmap 寫著：『NVIDIA Kyber will scale up into a massive all-to-all NVL1152 supercomputer using similar direct optical interconnects for rack-to-rack scale-up』；但 Jensen 在 Financial Analyst Q+A 又說 Feynman NVL1152 會是『all CPO』。目前仍存在分歧：rack 內 scale-up 是否繼續使用 copper，還是 CPO 會連 rack 內也一併取代。

Atomic Claim 237/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0195

Claim: 進入 Feynman generation 後，CPO 的使用會擴展到另一個大型 world-size rack：NVL1152，由 8 個 Kyber rack 組成。
Frame: NARY_RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 238/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0192::SPLIT02

Claim: Jensen Huang 在 Financial Analyst Q+A 表示，Feynman 的 NVL1152 將會是「全 CPO」。
Frame: NARY_RELATION · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 239/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0197

Claim: 目前對 rack 內 scale-up 到底會繼續使用 copper，還是由 CPO 取代 copper，存在不同說法。
Frame: COMPARISON · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 240/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0192

Claim: Nvidia Technical Blog 指出，Kyber 將透過類似的直接光學機櫃間互連擴展成大型 all-to-all NVL1152 超級電腦。
Frame: NARY_RELATION · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Nvidia’s approach has been to use copper where they can, and optics where they must. The architecture of NVL1152 in the Feynman generation will follow the same principle. It is clear that the NVL1152 will adopt CPO to connect between racks, but from GPUs to NVLink Switches is currently copper POR. Nvidia is unable to achieve another doubling of electrical lane speed from 224Gbit/s bi-di to 448Gbit/s uni-di means bandwidth isn’t that amazing.

Nvidia 一貫策略是『能用 copper 就用 copper，非用 optics 不可才用 optics』，Feynman NVL1152 也會遵循同樣原則。Rack-to-rack 已確定會導入 CPO；但目前 GPU 到 NVLink Switch 的 plan-of-record 仍是 copper。Nvidia 無法再把 electrical lane speed 從 224Gbit/s bi-di 直接翻倍到 448Gbit/s uni-di，因此 bandwidth 提升沒有想像中驚人。

Atomic Claim 241/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0199::RESEG01

Claim: Nvidia 的 networking approach 是只有在不得不用的地方才使用 optics。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 242/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0203

Claim: Nvidia 無法再把 electrical lane speed 從 224Gbit/s bi-di 翻倍到 448Gbit/s uni-di，因此可提供的 bandwidth 並沒有大幅提升。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 243/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0202

Claim: In SemiAnalysis’s March 2026 Feynman NVL1152 base case, the current GPU-to-NVLink-switch plan of record uses copper.
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 244/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0200

Claim: The Feynman-generation NVL1152 architecture will use copper where possible and optics where required.
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 245/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0201

Claim: 可以確定 NVL1152 將採用 CPO 進行機櫃間連接。
Frame: RELATION · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 246/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0198

Claim: Nvidia 一貫的原則，是能使用 copper 的地方就優先使用銅互連。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

While 448G high speed SerDes have big challenges for shoreline, reach, and power versus using a die-to-die connection to an optical engine, the manufacturing challenges, cost, and reliability for Feynman necessitate using copper to the Switch.

448G high-speed SerDes 相較 die-to-die connection + optical engine，在 shoreline、reach、power 上都有很大挑戰；但 Feynman 若考量 manufacturing difficulty、cost、reliability，仍迫使 Nvidia 在 GPU-to-switch 這段優先使用 copper。

Atomic Claim 247/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0204::SPLIT01

Claim: 448G high-speed SerDes 相較以 die-to-die 方式連到 optical engine，在 shoreline、reach 與 power 上面臨很大挑戰。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 248/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0204::SPLIT02

Claim: 考量 Feynman 的製造難度、成本與可靠性，系統仍需要使用 copper 連到 switch。
Frame: RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

With that said, the NVL1152 SKU is years out – and the roadmap is highly likely to shift. For now, our base case stands at copper being used within each rack and CPO between the racks, but this could easily change.

當然，NVL1152 SKU 還要幾年才會真正出現，roadmap 很可能繼續變。目前我們的 base case 還是 rack 內 copper、rack 間 CPO，但這完全可能改變。

Atomic Claim 249/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0207::SPLIT02

Claim: SemiAnalysis 對 NVL1152 的目前 base case，是機櫃之間使用 CPO。
Frame: RELATION · Mode: INFERRED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 250/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0206

Claim: SemiAnalysis expects the Feynman NVL1152 roadmap to change before launch.
Frame: CLAIM_ONLY · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 251/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0205

Claim: As of the March 2026 article, the Feynman NVL1152 SKU remains several years from launch.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 252/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0207::SPLIT01

Claim: SemiAnalysis 對 NVL1152 的目前 base case，是每個機櫃內使用 copper。
Frame: RELATION · Mode: INFERRED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 253/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0208

Claim: The March 2026 Feynman NVL1152 copper-within-rack/CPO-between-racks base case may change.
Frame: CLAIM_ONLY · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟本版 Claim 與證據

For now – our best estimate of Nvidia’s CPO roadmap is as follows:

目前我們對 Nvidia CPO roadmap 的最佳估計如下：

Rubin:

NVL72 – Oberon all copper scale up

NVL72：Oberon，全 copper scale-up。

Atomic Claim 254/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0210::REPAIR01

Claim: In the source Rubin roadmap section, NVL72 uses Oberon with all-copper scale-up.
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Rubin Ultra:

NVL72 – Oberon all copper scale up

NVL72：Oberon，全 copper scale-up。

Atomic Claim 255/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0211::REPAIR01

Claim: In the source Rubin Ultra roadmap section, NVL72 uses Oberon with all-copper scale-up.
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

NVL144 – Kyber rack all copper scale up

NVL144：Kyber rack，全 copper scale-up。

Atomic Claim 256/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0212

Claim: In SemiAnalysis’s Rubin Ultra roadmap estimate, NVL144 uses the Kyber form factor with all-copper scale-up.
Frame: NARY_RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

NVL288 – Kyber rack all copper scale up with copper connecting 2 racks together

NVL288：Kyber rack，全 copper scale-up，並用 copper 把兩個 rack 互連。

Atomic Claim 257/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0213::SPLIT01

Claim: For the Rubin Ultra roadmap estimate: NVL288 採用 Kyber rack form factor。
Frame: RELATION · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 258/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0213::SPLIT03

Claim: For the Rubin Ultra roadmap estimate: NVL288 以 copper 連接兩個機櫃。
Frame: RELATION · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 259/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0213::SPLIT02

Claim: For the Rubin Ultra roadmap estimate: NVL288 的 scale-up 全部使用 copper。
Frame: RELATION · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

NVL576 – 8x Oberon Racks copper scale up within rack and CPO on switch between racks in a two tier all to all topology. This would be low volume for test purposes

NVL576：由 8 個 Oberon rack 組成；rack 內使用 copper scale-up，rack 間 switch 使用 CPO，以 two-tier all-to-all topology 互連。這個版本預計只會低量用於 testing。

Atomic Claim 260/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0214::SPLIT04

Claim: For the Rubin Ultra roadmap estimate: NVL576 採用 two-tier all-to-all topology。
Frame: ATTRIBUTE · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 261/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0214::SPLIT01

Claim: For the Rubin Ultra roadmap estimate: NVL576 由 8 個 Oberon racks 組成。
Frame: RELATION · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 262/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0214::SPLIT02

Claim: For the Rubin Ultra roadmap estimate: NVL576 的 rack 內 scale-up 使用 copper。
Frame: RELATION · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 263/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0215

Claim: SemiAnalysis 預期 Rubin Ultra NVL576 的八個 Oberon rack、rack 間 CPO 配置只會用於低量測試。
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 264/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0214::SPLIT03

Claim: For the Rubin Ultra roadmap estimate: NVL576 的 rack 間 scale-up 在 switch 上使用 CPO。
Frame: RELATION · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Feynman:

NVL72 – Oberon Rack – All Copper

NVL72：Oberon Rack，全 copper。

Atomic Claim 265/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0216

Claim: Feynman NVL72 採 Oberon rack，全部使用 Copper。
Frame: RELATION · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

NVL144 – Kyber Rack – All Copper

NVL144：Kyber Rack，全 copper。

Atomic Claim 266/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0217

Claim: Feynman NVL144 採 Kyber Rack，全部使用 Copper。
Frame: NARY_RELATION · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

NVL1152 – 8xKyber Rack – Copper within rack and CPO on the switch between racks

NVL1152：8 個 Kyber Rack；rack 內使用 copper，rack 間 switch 使用 CPO。

Atomic Claim 267/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0218::SPLIT03

Claim: Feynman NVL1152 在機櫃之間的 switch 上使用 CPO。
Frame: RELATION · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 268/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0218::SPLIT01

Claim: Feynman NVL1152 由 8 個 Kyber racks 組成。
Frame: RELATION · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 269/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0218::SPLIT02

Claim: Feynman NVL1152 在機櫃內使用 copper。
Frame: RELATION · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: SemiAnalysis, Nvidia

Oberon and Kyber Updates, Larger World Sizes Introduced, More Networking Updates

Nvidia provided a long-awaited update on its Kyber rack form factor, the latest addition to the lineup after Oberon having first been previewed as a prototype at GTC 2025. As a prototype, the rack architecture has continued to evolve, and we notice some changes. First, each compute blade has densified, with 4x Rubin Ultra GPU and 2x Vera each. There are a total of 2 canisters of 18 compute blades which amounts to 36 compute blades total for 144 GPUs in a rack. The initial Kyber design featured 2 GPUs and 2 Vera CPUs in one compute blade, with a total of 4 canisters of 18 compute blades each.

Nvidia 終於更新了大家等很久的 Kyber rack form factor。Kyber 是 Oberon 之後最新加入 lineup 的 rack architecture，最早在 GTC 2025 以 prototype 形式亮相。既然是 prototype，architecture 一直持續演進，現在已看到幾個改變。第一，compute blade density 提高：每 blade 改成 4×Rubin Ultra GPU + 2×Vera。整 rack 有兩個 canister、每個 18 個 compute blade，共 36 個 blade、144 顆 GPU。最初 Kyber design 則是每 compute blade 2 顆 GPU + 2 顆 Vera CPU，總共四個 canister、每個 18 blade。

Atomic Claim 270/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0219::SPLIT03

Claim: Kyber was first previewed as a prototype at GTC 2025.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 271/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0223::SPLIT03

Claim: 最初的 Kyber design，整櫃有 4 個 canister，每個 canister 配置 18 個 compute blade。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 272/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0219::SPLIT02

Claim: Kyber 是繼 Oberon 之後加入產品線的最新 form factor。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 273/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0223::SPLIT01

Claim: 最初的 Kyber design，在每個 compute blade 中配置 2 顆 GPU。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 274/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0219::SPLIT01

Claim: Nvidia 更新了 Kyber rack form factor。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 275/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0222

Claim: In the March 2026 Kyber prototype/design discussion: 每個 rack 有 2 個 canister，每個 canister 含 18 個 compute blade，因此共 36 個 compute blade、合計 144 顆 GPU。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 276/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0221::SPLIT01

Claim: In the March 2026 Kyber prototype/design discussion: 每個 compute blade 配置 4 顆 Rubin Ultra GPU。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 277/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0221::SPLIT02

Claim: In the March 2026 Kyber prototype/design discussion: 每個 compute blade 配置 2 顆 Vera CPU。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 278/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0223::SPLIT02

Claim: 最初的 Kyber design，在每個 compute blade 中配置 2 顆 Vera CPU。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

The details below are based on the Rubin Kyber prototypes, but Rubin Ultra will be redone.

下面細節基於 Rubin Kyber prototype，但 Rubin Ultra 版本還會重新設計。

Atomic Claim 279/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0225

Claim: 原文預期：Rubin Ultra 的設計將重新調整。
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: SemiAnalysis

Each switch blade is also double in height vs the GTC 2025 prototype, with 6 NVLink 7 switches per switch blade, and 12 switch blades per rack, amounting to a total of 72 NVLink 7 switches per Kyber rack. The GPUs are connected all-to-all to the switch blades via 2 PCB midplanes or 1 midplane per canister.

每個 switch blade 相較 GTC 2025 prototype 高度也加倍，每 blade 放 6 顆 NVLink 7 switch，整 rack 有 12 個 switch blade，因此 Kyber rack 總計 72 顆 NVLink 7 switch。GPU 透過兩塊 PCB midplane 和 switch blade 做 all-to-all connection，也就是每個 canister 配一塊 midplane。

Atomic Claim 280/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0226::SPLIT04

Claim: In the March 2026 Kyber prototype/design discussion: 每個 Kyber rack 總共有 72 顆 NVLink 7 switch。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 281/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0227::SPLIT01

Claim: In the March 2026 Kyber prototype/design discussion: 在 Kyber 中，GPU 會以 all-to-all 方式連到 switch blade。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 282/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0226::SPLIT01

Claim: Kyber 的每個 switch blade 高度比 GTC 2025 prototype 增加一倍。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 283/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0227::SPLIT02

Claim: In the March 2026 Kyber prototype/design discussion: Kyber 的 GPU-to-switch-blade 連線透過 2 片 PCB midplane，等價於每個 canister 1 片 midplane。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 284/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0226::SPLIT02

Claim: In the March 2026 Kyber prototype/design discussion: Kyber 的每個 switch blade 配置 6 顆 NVLink 7 switch。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 285/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0226::SPLIT03

Claim: In the March 2026 Kyber prototype/design discussion: 每個 Kyber rack 配置 12 個 switch blade。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Kyber midplane PCB (GPU side). Source: Nvidia, SemiAnalysis

Kyber midplane PCB（GPU side）。Source: Nvidia, SemiAnalysis

For Rubin Ultra NVL144 Kyber, there will be no CPO used for scale up as we have told clients multiple times ↗, despite rumors from other analysts suggesting scale-up CPO introduction for Kyber. However, optics for NVLink are coming and will be progressively phased in. Scale-up CPO will first be used for the Rubin Ultra NVL 576 system to connect between 8 Oberon form factor racks, forming a two-layer all-to-all network. A copper backplane will still be used for scale-up networking within the racks however. This is still for low volume / testing purposes.

Rubin Ultra NVL144 Kyber 不會在 rack 內 scale-up 使用 CPO，這點我們已多次告訴 client ↗，儘管其他 analyst 一直有傳言說 Kyber 會導入 scale-up CPO。不過 NVLink optics 的確會來，而且會逐步導入。Scale-up CPO 首先會用在 Rubin Ultra NVL576：把 8 個 Oberon form-factor rack 互連，形成 two-layer all-to-all network；rack 內 scale-up 仍使用 copper backplane。這個版本目前仍以低量／testing 為主。

Atomic Claim 286/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0231

Claim: 不過，NVLink 的光學連線即將導入。
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 287/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0233::SPLIT01

Claim: scale-up CPO 會先用在 Rubin Ultra NVL576。
Frame: NARY_RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 288/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0233::SPLIT02

Claim: Rubin Ultra NVL576 連接 8 個 Oberon form-factor rack。
Frame: RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 289/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0230::SPLIT01

Claim: 對 Rubin Ultra NVL144 Kyber 而言，scale-up 不會使用 CPO。
Frame: NARY_RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 290/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0230::SPLIT02

Claim: 其他分析師曾傳出 Kyber 將導入 scale-up CPO 的消息。
Frame: NARY_RELATION · Mode: RUMORED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 291/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0235

Claim: SemiAnalysis 預期 Rubin Ultra NVL576 的 inter-rack CPO 配置仍僅用於低量／測試用途。
Frame: CLAIM_ONLY · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 292/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0233::SPLIT03

Claim: Rubin Ultra NVL576 的 scale-up network 採 two-layer all-to-all topology。
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 293/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0230

Claim: NVLink optics 將逐步分階段導入。
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 294/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0234

Claim: In the described Rubin Ultra NVL576 roadmap, scale-up within each Oberon rack retains a copper backplane.
Frame: NARY_RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Moving back to the Kyber Rack, each Rubin Ultra logical GPU offers 14.4Tbit/s uni-di of scale-up bandwidth, using an 80DP connector (72 DPs used x 200Gbit/s bi-di channel = 14.4Tbit/s) per GPU for connectivity to the midplane board. Connecting all 144 GPUs in an all-to-all network will require 72 NVLink 7.0 Switch Chips running at 28.8Tbit/s uni-di of aggregate bandwidth each.

回到 Kyber Rack，每顆 Rubin Ultra logical GPU 提供 14.4Tbit/s uni-di scale-up bandwidth，透過每 GPU 一個 80DP connector 連到 midplane board；其中使用 72 DP × 200Gbit/s bi-di channel = 14.4Tbit/s。若要把 144 顆 GPU 全部連成 all-to-all network，需要 72 顆 NVLink 7.0 Switch Chip，每顆 aggregate bandwidth 28.8Tbit/s uni-di。

Atomic Claim 295/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0236::SPLIT04

Claim: In the March 2026 Kyber prototype/design discussion: In the updated Rubin Ultra Kyber design, each GPU connector joins the GPU to the midplane board.
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 296/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0236::SPLIT02

Claim: In the March 2026 Kyber prototype/design discussion: In the updated Kyber design, each Rubin Ultra GPU uses one 80-DP midplane connector with 72 DPs active.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 297/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0236::SPLIT03

Claim: In the March 2026 Kyber prototype/design discussion: In the updated Rubin Ultra Kyber design, 72 active DPs at 200Gbit/s simultaneous bidirectional channel signaling correspond to the stated 14.4Tbit/s unidirectional GPU bandwidth.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 298/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0236::SPLIT01

Claim: In the March 2026 Kyber prototype/design discussion: 每顆 Rubin Ultra logical GPU 可提供 14.4Tbit/s uni-di scale-up bandwidth。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 299/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0237::SPLIT01

Claim: In the March 2026 Kyber prototype/design discussion: 要把 144 顆 GPU 以 all-to-all network 互連，需要 72 顆 NVLink 7.0 switch chips。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 300/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0237::SPLIT02

Claim: In the March 2026 Kyber prototype/design discussion: 每顆 NVLink 7.0 switch chip 提供 28.8Tbit/s uni-di aggregate bandwidth。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: SemiAnalysis

In the Kyber Switch Blade picture below, we can see that there are 2 separate PCBs carrying 3 Switches each. The switch blade should have 6 152DP connectors, 3 connectors serving each midplane board. The picture is a prototype blade using less dense connectors, which is why there are 12 connectors instead of the 6 that we expect in the production version.

從下方 Kyber Switch Blade 圖可以看到，實際上有兩塊獨立 PCB，每塊各放 3 顆 Switch。Production switch blade 應會有 6 個 152DP connector，每三個 connector 對應一塊 midplane board。照片中的 prototype blade 使用較低 density connector，所以才會看到 12 個 connector，而不是我們預期 production version 的 6 個。

Atomic Claim 301/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0238

Claim: In the March 2026 Kyber prototype/design discussion: 從 Kyber switch-blade 圖片可看到兩片獨立 PCB，每片承載 3 顆 switch。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 302/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0240

Claim: The Kyber switch-blade prototype has 12 lower-density connectors; the source expects six connectors in the production design.
Frame: CLAIM_ONLY · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 303/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0239

Claim: In the March 2026 Kyber prototype/design discussion: 原文預期：switch blade 預計會有 6 個 152DP connector，其中每個 midplane board 由 3 個 connector 服務。
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Source: Nvidia, SemiAnalysis

Each 28.8T NVLink Switch has 144 lanes of 200G (simultaneous bi-directional) which means each Switch has 24 lanes of 200G going to each connector. Copper flyover cables are used to connect each switch to the midplane, as the distances involved are too long for PCB traces. This is also why the switches are further away from the midplane, to provide space for the routing of the flyover cables.

每顆 28.8T NVLink Switch 有 144 lanes × 200G simultaneous bi-directional，因此每顆 Switch 會分 24 條 200G lane 到每個 connector。Switch 到 midplane 使用 copper flyover cable，因為距離太長，PCB trace 無法承受。這也是為什麼 switch 會刻意離 midplane 遠一點，以保留 flyover cable routing 空間。

Atomic Claim 304/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0242

Claim: In the March 2026 Kyber prototype/design discussion: In the described Kyber switch blade, copper flyover cables join each NVLink switch to the midplane because the path is too long for PCB traces.
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 305/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0241

Claim: In the March 2026 Kyber prototype/design discussion: 每顆 28.8T NVLink Switch 有 144 lanes 的 200G simultaneous bi-directional link，因此每顆 switch 會有 24 lanes 的 200G 連到每個 connector。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 306/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0243

Claim: In the March 2026 Kyber prototype/design discussion: In the described Kyber switch blade, switches sit farther from the midplane to leave room for flyover cable routing.
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Source: SemiAnalysis Networking Model ↗

Each NVLink Switch Chip connects via flyover cables to the connector (144 DPs used x 200 Gbit/s bi-di channel = 28.8Tbit/s) connectors at the edge of the switch blade, and these connectors plug into the midplane board. Nvidia is looking into using co-packaged Copper to reduce loss further, in case NPC doesn’t work. As far as we know the Nvidia is telling supply chain to go for fully co-packaged copper.

每顆 NVLink Switch Chip 透過 flyover cable 連到 switch blade 邊緣 connector；完整使用 144 DP × 200Gbit/s bi-di channel = 28.8Tbit/s。這些 connector 再插入 midplane board。Nvidia 也在研究 co-packaged copper，以便如果 NPC 不可行時進一步降低 loss；就我們所知，Nvidia 現在給 supply chain 的方向是全面朝 co-packaged copper。

Atomic Claim 307/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0244

Claim: In the March 2026 Kyber prototype/design discussion: 每顆 NVLink Switch chip 透過 flyover cable 連到 switch blade 邊緣的 connector（使用 144 DPs × 200Gbit/s bi-di＝28.8Tbit/s），再由 connector 插入 midplane board。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 308/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0246

Claim: In the March 2026 Kyber prototype/design discussion: 依 SemiAnalysis 目前掌握，Nvidia 正要求 supply chain 朝 fully co-packaged copper 的方向準備。
Frame: RELATION · Mode: ATTRIBUTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 309/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0245

Claim: In the March 2026 Kyber prototype/design discussion: Nvidia 正在評估使用 co-packaged Copper 進一步降低 loss，以防 NPC 無法運作。
Frame: RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Rubin Ultra NVL288

Though not officially discussed by Nvidia at GTC 2026, an NVL288 concept has been explored within the supply chain. This would entail two NVL144 Kyber racks placed adjacent to each other, with a rack-to-rack copper backplane used to connect the two racks. One possibility is that all 288 GPUs are connected all to all, but this would require higher radix switches than the current NVLink 7 switches which only offer a maximum radix of 144 ports of 200G.

Nvidia 雖沒有在 GTC 2026 正式談 NVL288，但 supply chain 內部確實探索過這個 concept：把兩個 NVL144 Kyber rack 並排，以 rack-to-rack copper backplane 互連。其中一種可能是讓 288 顆 GPU 全部 all-to-all，但這需要比現有 NVLink 7 更高 radix 的 switch，因為目前 NVLink 7 maximum radix 只有 144 個 200G port。

Atomic Claim 310/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0249

Claim: 在尚未正式公布的雙 Kyber rack NVL288 構想中，可能以 rack-to-rack copper backplane 連接兩個 rack。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 311/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0248

Claim: 尚未正式公布的 NVL288 構想可能將兩個 NVL144 Kyber racks 並排放置。
Frame: ATTRIBUTE · Mode: HYPOTHETICAL · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 312/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0251

Claim: 在原文所述假設情境中，這種方案需要比目前 NVLink 7 更高 radix 的 switch，因為現有 NVLink 7 最大只提供 144 個 200G port。
Frame: ATTRIBUTE · Mode: HYPOTHETICAL · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 313/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0250

Claim: 在原文所述假設情境中，其中一種可能方案，是讓全部 288 顆 GPUs 彼此 all-to-all 連接。
Frame: ATTRIBUTE · Mode: HYPOTHETICAL · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 314/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0247

Claim: 雖然 Nvidia 並未在 GTC 2026 正式討論，但 supply chain 內部已探索過 NVL288 concept。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

If Rubin Ultra NVL288 is deployed, each Rubin Ultra GPU will have a scale-up bandwidth of 14.4Tbit/s uni-di, requiring 144 DPs of cables to connect the NVLink 7 switches. 72 DPs per GPU times 288 GPUs means a total of 20,736 additional DPs required to connect this larger world size domain. This entails a lot of cables, so it is an upper bound of how much cable content could be used.

如果 Rubin Ultra NVL288 真正部署，每顆 Rubin Ultra GPU 仍有 14.4Tbit/s uni-di scale-up bandwidth，需要 144 DP cable 連 NVLink 7 switch。72 DP/GPU × 288 GPU，總計需要額外 20,736 DP 才能支援這個更大 world-size domain。Cable 數量非常驚人，因此這可以視為 copper content 的上限情境。

Atomic Claim 315/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0252

Claim: 若部署 Rubin Ultra NVL288，每顆 Rubin Ultra GPU 會有 14.4Tbit/s uni-di scale-up bandwidth，並需要 144 DPs cable 連接 NVLink 7 switch。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 316/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0253

Claim: 若部署 Rubin Ultra NVL288，原文以每 GPU 額外 72 DPs × 288 GPUs 計算，較大 world-size domain 需要額外 20,736 DPs；這是可能 cable content 的上界情境。
Frame: ATTRIBUTE · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 317/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0254

Claim: The article treats the NVL288 scenario’s additional 20,736 cable DPs as an upper-bound estimate.
Frame: CLAIM_ONLY · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟本版 Claim 與證據

The radix of the 28.8T NVLink Switch limits the number of GPUs that each switch can connect while still providing for cross-rack connectivity. Either a higher radix switch will have to be used - or there will have to be a degree of oversubscription in this architecture while potentially adopting a dragonfly-like network topology. This would also require fewer DPs worth of copper cables.

28.8T NVLink Switch 的 radix 會限制每顆 switch 能連多少 GPU，同時還要保留 cross-rack connectivity。解法不是換更高 radix switch，就是 architecture 必須接受一定程度 oversubscription，甚至改採類 dragonfly network topology；後者也會減少需要的 copper cable DP 數量。

Atomic Claim 318/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0256

Claim: For the hypothetical NVL288 concept, the article considers either higher-radix switches or oversubscription with a possible dragonfly-like topology.
Frame: CLAIM_ONLY · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 319/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0255

Claim: 在原文所述假設情境中，28.8T NVLink Switch 的 radix 會限制每顆 switch 能連接的 GPU 數量，同時還必須保留 cross-rack connectivity。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 320/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0257

Claim: 在可能採用 oversubscription／dragonfly-like topology 的 NVL288 替代方案中，所需 copper cable DPs 會較少。
Frame: ATTRIBUTE · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟本版 Claim 與證據

Source: SemiAnalysis

All current evidence in the supply chain points to NVSwitch 7 being the same bandwidth as NVSwitch 6, but that is seems a bit illogical to be frank. Our belief is that NVSwitch 7 is actually 2x the bandwidth and radix of NVSwitch 6, so all-to-all can be done, and architecturally that makes the most sense from a systems perspective.

目前 supply chain 所有 evidence 都指向 NVSwitch 7 bandwidth 和 NVSwitch 6 相同，但坦白說這有點不合理。我們更傾向認為 NVSwitch 7 實際上會有 NVSwitch 6 的 2x bandwidth、2x radix，這樣才有辦法做 all-to-all；從 system architecture 的角度看也最合理。

Atomic Claim 321/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0261::RESEG01

Claim: SemiAnalysis 判斷 NVSwitch 7 的 radix 可能是 NVSwitch 6 的 2 倍。
Frame: CLAIM_ONLY · Mode: INFERRED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 322/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0261::RESEG02

Claim: 在 SemiAnalysis 對 NVSwitch 7 具有 2 倍 bandwidth 與 2 倍 radix 的假設下，scale-up network 可以實現 all-to-all connectivity。
Frame: CLAIM_ONLY · Mode: INFERRED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 323/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0261::RESEG03

Claim: SemiAnalysis 認為，NVSwitch 7 採 2 倍 bandwidth 與 2 倍 radix、以支援 all-to-all 的 architecture，從 system perspective 看最合理。
Frame: CLAIM_ONLY · Mode: INFERRED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 324/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0260::RESEG01

Claim: SemiAnalysis 判斷 NVSwitch 7 的 bandwidth 可能是 NVSwitch 6 的 2 倍。
Frame: CLAIM_ONLY · Mode: INFERRED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 325/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0258

Claim: SemiAnalysis reports that current supply-chain evidence points to NVSwitch 7 having the same bandwidth as NVSwitch 6, while separately stating an inference that it may double.
Frame: COMPARISON · Mode: ATTRIBUTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Rubin Ultra NVL576

To push the scale up world size beyond 144 GPUs and across multiple racks, optics are needed as we are approaching the maximum compute density that is within the reach of copper. Rubin Ultra NVL576 is now on the roadmap with 8 racks of lower density Oberon.

要把 scale-up world size 從 144 GPU 再往上推、跨越多個 rack，就需要 optics，因為 compute density 已接近 copper reach 的極限。Rubin Ultra NVL576 現在已在 roadmap 上，由 8 個較低 density 的 Oberon rack 組成。

Atomic Claim 326/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0264::SPLIT01

Claim: Rubin Ultra NVL576 現已列入 roadmap。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 327/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0264::SPLIT02

Claim: Rubin Ultra NVL576 配置 8 個較低密度的 Oberon racks。
Frame: RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 328/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0260

Claim: 若要把 scale-up world size 擴展到超過 144 顆 GPUs 並跨越多個機櫃，就需要 optics，因為系統已接近 copper 可涵蓋的最大運算密度。
Frame: ATTRIBUTE · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟本版 Claim 與證據

Source: SemiAnalysis

Optics will be required for the inter-rack connections, though strictly speaking it isn’t confirmed whether this will be with pluggable optics or with CPO, though CPO seems much more likely. The current Blackwell NVL576 prototype “Polyphe” uses pluggable optics.

Rack 間 connection 一定需要 optics；嚴格來說，目前還沒正式確認會用 pluggable optics 還是 CPO，但 CPO 看起來明顯更可能。目前 Blackwell NVL576 prototype『Polyphe』使用的是 pluggable optics。

Atomic Claim 329/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0265::SPLIT01

Claim: Rubin Ultra NVL576 is expected to require optical inter-rack connections.
Frame: CLAIM_ONLY · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 330/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0266

Claim: 目前 Blackwell NVL576 prototype「Polyphe」使用 pluggable optics。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 331/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0265::SPLIT03

Claim: SemiAnalysis considers CPO much more likely than pluggables for Rubin Ultra NVL576 inter-rack optics.
Frame: COMPARISON · Mode: INFERRED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 332/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0265::SPLIT02

Claim: The March 2026 article says the choice between pluggables and CPO for Rubin Ultra NVL576 inter-rack optics is not confirmed.
Frame: COMPARISON · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

We have shown a concept of NVL576 for GB200 previously ↗ with pluggable optics to interconnect the second layer of NVLink switches. The use of pluggables contributed to an enormous increase in BOM cost that made the system untenable from a TCO perspective for a switched all-to-all. However, it is plausible that Rubin Ultra NVL576 will be rolled out in test volumes before Feynman NVL 1,152, where we will see actual volume ramp of scale-up CPO.

我們之前已展示過 GB200 NVL576 concept ↗，以 pluggable optics 連接第二層 NVLink switch。Pluggable 帶來非常巨大的 BoM cost increase，讓 switched all-to-all system 從 TCO 角度幾乎不可行。不過 Rubin Ultra NVL576 很可能仍會先用 test volume 推出，替之後 Feynman NVL1152 真正大量 ramp scale-up CPO 做準備。

Atomic Claim 333/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0269::EPI02

Claim: In the source plausible roadmap scenario, actual scale-up CPO volume ramp occurs with Feynman NVL1152 after Rubin Ultra NVL576 test volumes.
Frame: ATTRIBUTE · Mode: HYPOTHETICAL · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 334/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0269::EPI01

Claim: SemiAnalysis considers it plausible that Rubin Ultra NVL576 will roll out in test volumes before Feynman NVL1152.
Frame: ATTRIBUTE · Mode: HYPOTHETICAL · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 335/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0268

Claim: In the previous GB200 NVL576 switched-all-to-all concept, pluggable optics raised BOM cost enough to make the design untenable on TCO.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

The downstream implications of this are exposed in our institutional research, trusted by all major hyperscalers, semiconductor companies, and AI Labs, at sales@semianalysis.com

這些 downstream implication 都收錄在我們的 institutional research 中，主要 hyperscaler、semiconductor company、AI Lab 都有使用；如需洽詢可聯絡 sales@semianalysis.com。

Feynman

While not much is known about Feynman, the Keynote sneak peek was enough to tell us Feynman will be exciting, with three major technical innovations all being pushed in a single platform: Hybrid bonding/SoIC ↗, A16, CPO ↗, and custom HBM ↗.

雖然現在對 Feynman 還知道不多，但 Keynote 的 sneak peek 已經足以看出它會很精彩：單一 platform 同時推進多項重大 technical innovation，包括 Hybrid bonding／SoIC ↗、A16、CPO ↗、custom HBM ↗。

Atomic Claim 336/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0267

Claim: Feynman 將採用 Hybrid bonding／SoIC。
Frame: NARY_RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 337/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0270

Claim: Feynman 將採用客製化 HBM。
Frame: RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 338/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0268

Claim: Feynman 將採用 A16。
Frame: RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 339/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0269

Claim: Feynman 將採用 CPO。
Frame: RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

While Feynman adopting CPO is on the roadmap, the question is to what extent? Will in-rack interconnectivity be copper based or optical? We will show possible configurations behind the Paywall. Vera ETL256

Feynman 導入 CPO 已經在 roadmap 上，真正問題是會導入到什麼程度：rack 內 interconnect 仍使用 copper，還是也會改成 optical？Paywall 後我們會展示幾種可能 configuration。接著先談 Vera ETL256。

Atomic Claim 340/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0274

Claim: In March 2026, the extent of copper versus optical connectivity inside Feynman racks remains unsettled.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 341/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0273

Claim: Feynman 採用 CPO 已在 roadmap 上，但實際導入範圍仍未確定。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

CPU demand is rising as AI workloads require more data handling, preprocessing, and orchestration beyond GPU compute. Reinforcement learning further increases demand, with CPUs running simulations, executing code, and verifying outputs in parallel. As GPUs scale faster than CPUs, larger CPU clusters are needed to keep them fully utilized, making CPUs a growing bottleneck.

AI workload 不只需要 GPU compute，還需要更多 data handling、preprocessing、orchestration，因此 CPU demand 正快速上升。Reinforcement learning 又會進一步放大 CPU 需求，因為 CPU 必須平行執行 simulation、code execution、output verification。當 GPU scale 速度快於 CPU，為了讓 GPU 保持 fully utilized，就需要更大的 CPU cluster，CPU 也逐漸成為新的 bottleneck。

Atomic Claim 342/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0278

Claim: 由於 GPU 的擴張速度快於 CPU，需要更大的 CPU cluster 才能維持 GPU 充分利用，使 CPU 成為日益明顯的 bottleneck。
Frame: COMPARISON · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 343/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0276

Claim: 隨著 AI workload 在 GPU compute 之外需要更多資料處理、preprocessing 與 orchestration，CPU demand 正在上升。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 344/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0277

Claim: Reinforcement learning 進一步推升 CPU demand，因為 CPU 會平行執行 simulation、code execution 與 output verification。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

The Vera standalone rack addresses this directly, achieving unprecedented density by fitting 256 CPUs into a single rack — a feat that necessitates liquid cooling. The underlying rationale mirrors the NVL rack design philosophy: pack compute tightly enough that copper interconnects can reach everything within the rack, eliminating the need for optical transceivers on the spine. The cost savings from copper more than offset the additional cooling overhead.

Vera standalone rack 就是直接針對這個問題：單一 rack 塞進 256 顆 CPU，density 前所未見，也因此必須 liquid cooling。背後邏輯和 NVL rack design philosophy 一樣——把 compute 壓得夠密，讓 rack 內所有 connection 都還在 copper reach 內，就可以拿掉 spine 上的 optical transceiver。Copper 帶來的 cost saving 足以抵銷額外 cooling overhead。

Atomic Claim 345/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0281

Claim: copper 帶來的成本節省，超過額外 cooling overhead。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 346/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0280

Claim: 其設計邏輯與 NVL rack 類似：把 compute 壓得足夠緊密，使 copper interconnect 能覆蓋整個 rack，從而不需要 spine 上的 optical transceivers。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 347/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0279::SPLIT01

Claim: Vera standalone rack 可在單一 rack 容納 256 顆 CPU。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 348/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0279::SPLIT02

Claim: Vera standalone rack 的 256-CPU 高密度配置需要 liquid cooling。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: SemiAnalysis

Each Vera ETL rack consists of 32 compute trays, 16 above and 16 below, arranged symmetrically around four 1U MGX ETL switch trays (based on Spectrum-6) in the middle. The symmetric split is deliberate: it minimizes cable length variance between compute trays and the spine, keeping all connections within copper reach. From each switch tray, rear-facing ports connect to that copper spine for intra-rack communication, while 32 front-facing OSFP cages provide optical connectivity to the rest of the POD.

每個 Vera ETL rack 有 32 個 compute tray，上下各 16 個，對稱排列在 rack 中央四個 1U MGX ETL switch tray 周圍；這些 switch tray 基於 Spectrum-6。對稱 layout 是刻意設計，用來縮小不同 compute tray 到 spine 的 cable length variance，確保所有 connection 都維持在 copper reach 內。每個 switch tray 後方 port 連到 copper spine，負責 intra-rack communication；前方則有 32 個 OSFP cage，提供到 POD 其他部分的 optical connectivity。

Atomic Claim 349/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0283::RESEG02

Claim: 每個 Vera ETL rack 有 4 個 MGX ETL switch trays。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 350/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0285

Claim: Each Vera ETL256 switch tray has rear-facing ports connected to the copper spine for intra-rack communication.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 351/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0284

Claim: The symmetric Vera ETL256 tray arrangement minimizes cable-length variance to the spine and keeps intra-rack links within copper reach.
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 352/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0283::RESEG05

Claim: Vera ETL rack 的 4 個 MGX ETL switch trays 位於 rack 中央。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 353/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0283

Claim: In the Vera ETL256 rack, 16 compute trays sit below the central switch trays.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 354/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0286

Claim: Each Vera ETL256 switch tray has 32 front-facing OSFP cages for optical links to the rest of the POD.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 355/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0281

Claim: Each Vera ETL256 rack contains 32 compute trays.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 356/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0283::RESEG04

Claim: Vera ETL rack 的 MGX ETL switch trays 是基於 Spectrum-6。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 357/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0283::RESEG03

Claim: Vera ETL rack 的 MGX ETL switch trays 為 1U form factor。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 358/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0282

Claim: In the Vera ETL256 rack, 16 compute trays sit above the central switch trays.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Networking within the rack uses a Spectrum-X multiplane topology, distributing 200 Gb/s lanes across the four switches to achieve full all-to-all connectivity while maintaining a single network tier. With each compute tray housing 8 Vera CPUs, the result is 256 CPUs per rack, all interconnected over Ethernet through a single, flat network.

Rack 內 networking 使用 Spectrum-X multi-plane topology，把 200Gb/s lane 分散到四台 switch，在維持 single network tier 的同時做到 full all-to-all connectivity。每個 compute tray 有 8 顆 Vera CPU，因此整 rack 共有 256 顆 CPU，全部透過單層、flat Ethernet network 互連。

Atomic Claim 359/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0287

Claim: Vera ETL256 distributes 200Gb/s lanes across four Spectrum-X switches for single-tier all-to-all connectivity.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 360/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0288::SPLIT03

Claim: Vera ETL rack 內的 CPU 透過 Ethernet 以單一 flat network 互連。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 361/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0288::SPLIT01

Claim: Each Vera ETL256 compute tray contains eight Vera CPUs.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 362/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0288::SPLIT02

Claim: 每個 Vera ETL rack 共配置 256 顆 CPU。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: SemiAnalysis

Source: Nvidia ↗

CMX and STX

We have written extensively on Nvidia’s CMX, or ICMS platform in our last Rubin piece and Memory Model. Nvidia introduced the STX reference storage rack architecture.

我們在上一篇 Rubin 文章與 Memory Model 已經大量討論 Nvidia CMX／ICMS platform。這次 Nvidia 又推出 STX reference storage rack architecture。

Atomic Claim 363/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0290

Claim: Nvidia 推出了 STX reference storage rack architecture。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

CMX

CMX is NVIDIA’s context memory storage platform. CMX addresses a growing bottleneck in modern inference infrastructure: the rapid expansion of KV Cache required to support long-context and agentic workloads.

CMX 是 NVIDIA 的 context-memory storage platform，針對 modern inference infrastructure 越來越大的 bottleneck：為了支援 long-context、agentic workload，所需 KV cache 正快速膨脹。

Atomic Claim 364/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0292

Claim: CMX 用來處理現代 inference infrastructure 日益明顯的 bottleneck：支援 long-context 與 agentic workloads 所需的 KV Cache 快速成長。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 365/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0291

Claim: CMX 是 NVIDIA 的 context memory storage platform。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

KV cache grows linearly with input sequence length and number of users and is the primary tradeoff when it comes to prefill performance (time to first token). At scale, on-device HBM does not have enough capacity. Host DRAM extends beyond HBM capacity with an additional tier of cache, but also hits limits on total amount per node, memory bandwidth, and network bandwidth. Enter NVMe storage for additional KVcache offload.

KV cache 會隨 input sequence length、user 數線性增加，也是 prefill performance（time-to-first-token）的主要 trade-off。Scale 上去後，on-device HBM capacity 不夠；host DRAM 可以在 HBM 外再加一層 cache，但每 node 可放的總量、memory bandwidth、network bandwidth 最後也會碰到限制。再下一層就是用 NVMe storage 做額外 KV cache offload。

Atomic Claim 366/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0296

Claim: Host DRAM 可在 HBM 之外增加一層 cache，進一步擴充容量。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 367/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0295::SPLIT02

Claim: Host DRAM 會受到每個 node 可配置總容量的限制。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 368/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0294

Claim: 在 prefill 效能（time to first token）方面，KV cache 是主要 trade-off。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 369/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0295::SPLIT04

Claim: Host DRAM 會受到 network bandwidth 限制。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 370/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0293::SPLIT01

Claim: KV cache 會隨輸入 sequence length 線性成長。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 371/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0293::SPLIT02

Claim: KV cache 會隨 user 數量線性成長。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 372/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0296

Claim: 因此可使用 NVMe storage，進一步 offload KV cache。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 373/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0294

Claim: 在大規模情境下，device 上的 HBM 容量不足。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 374/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0295::SPLIT03

Claim: Host DRAM 會受到 memory bandwidth 限制。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

NVIDIA introduced a “new” intermediate storage “tier G3.5” within the inference memory hierarchy at CES in January. Tier G3.5 NVMe sits in between tier G3 DRAM and tier G4 shared storage (also NVMe, or SATA/SAS SSD, or HDD). Previously referred to as ICMS (Inference Context Memory Storage) and now branded as the CMX platform, this is just another re-brand of storage servers attached to compute servers via Bluefield NICs. The only difference from NVMe architectures is the swap from Connect-X NICs to Bluefield NICs.

NVIDIA 在今年 1 月 CES 引入一個『新』的中間 storage tier——G3.5，位於 inference memory hierarchy 的 tier G3 DRAM 與 tier G4 shared storage 之間；tier G4 本身可以是 NVMe、SATA/SAS SSD 或 HDD。這套東西原本叫 ICMS（Inference Context Memory Storage），現在 rebrand 成 CMX platform，本質上仍是 storage server 透過 BlueField NIC 接到 compute server。和其他 NVMe architecture 的主要差異，只是把 ConnectX NIC 換成 BlueField NIC。

Atomic Claim 375/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0300::SPLIT01

Claim: 這套平台先前稱為 ICMS，現在改稱 CMX。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 376/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0299::RESEG03

Claim: tier G4 shared storage 可以使用 HDD。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 377/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0299::RESEG01

Claim: tier G4 shared storage 可以使用 NVMe。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 378/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0299

Claim: NVIDIA 在 1 月 CES 發表了推論記憶體階層中的「新」中間 storage tier「G3.5」。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 379/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0297::SPLIT02

Claim: Tier G3.5 的技術為 NVMe。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 380/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0300::SPLIT02

Claim: 原文將 CMX 描述為透過 BlueField NICs 連接 compute servers 的 storage-server architecture。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 381/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0298::RESEG01

Claim: The March article’s body calls local NVMe G3.5 between G3 DRAM and G4 shared storage; its embedded Nvidia figure instead labels DRAM G2 and local SSD G3.
Frame: CLAIM_ONLY · Mode: ATTRIBUTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 382/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0299::RESEG02

Claim: tier G4 shared storage 可以使用 SATA／SAS SSD。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 383/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0301

Claim: 相較既有 NVMe architecture，原文稱唯一差異是把 Connect-X NIC 換成 BlueField NIC。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 384/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-G3-LAT

Claim: The cited Nvidia hierarchy illustration places G3 local SSD/rack-local warm KV reuse in the microsecond range.
Frame: ATTRIBUTE · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 385/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-G2-LAT

Claim: The cited Nvidia hierarchy illustration labels G2 system DRAM staging/spillover KV access at about 10–100ns.
Frame: ATTRIBUTE · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 386/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-G4-LAT

Claim: The cited Nvidia hierarchy illustration places G4 shared object/file cold or shared KV context in the millisecond range.
Frame: ATTRIBUTE · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: Original NVIDIA ICMS blog in January, 2026 – updated and re-released on March 16, 2026 https://developer.[[02_companies/NVDA|nvidia]].com/blog/introducing-[[02_companies/NVDA|nvidia]]-[[04_knowledge_base/BlueField-4|bluefield-4]]-powered-inference-context-memory-storage-platform-for-the-next-frontier-of-ai/ ↗

STX

To expand the scope of CMX, NVIDIA also launched STX. STX is a reference rack architecture using Nvidia’s BF-4 based storage solution to complement VR compute racks. The reference architecture effectively specifies exactly how many drives, Vera CPUs, BF-4 DPUs, CX-9 NICs, and Spectrum-X switches are needed for a given cluster.

為擴大 CMX scope，NVIDIA 又推出 STX。STX 是一套 reference rack architecture，使用 Nvidia BF-4-based storage solution 搭配 VR compute rack。Reference architecture 幾乎直接規定一個 cluster 應該配置多少 drive、Vera CPU、BF-4 DPU、CX-9 NIC、Spectrum-X switch。

Atomic Claim 387/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0304::SPLIT04

Claim: STX reference architecture 會明確規定一個 cluster 需要多少 CX-9 NICs。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 388/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0304::SPLIT02

Claim: STX reference architecture 會明確規定一個 cluster 需要多少 Vera CPUs。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 389/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0305

Claim: The STX reference architecture specifies the Spectrum-X switches required for a given cluster.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 390/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0302

Claim: 為擴大 CMX 的範圍，NVIDIA 也推出 STX。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 391/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0304::SPLIT01

Claim: STX reference architecture 會明確規定一個 cluster 需要多少 drives。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 392/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0304::SPLIT03

Claim: STX reference architecture 會明確規定一個 cluster 需要多少 BF-4 DPUs。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 393/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0303::SPLIT03

Claim: STX 用來補充 VR compute racks。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 394/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0303::SPLIT01

Claim: STX 是 reference rack architecture。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 395/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0303::SPLIT02

Claim: STX 採用 Nvidia 的 BF-4 based storage solution。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

BF-4 in STX. Source: Nvidia, SemiAnalysis

STX 中的 BF-4。Source: Nvidia, SemiAnalysis

Unlike the BF-4 in the VR NVL72, which consists of a Grace CPU and a single CX-9 NIC, the BF-4 in the STX reference design includes one Vera CPU, two CX-9 NICs, and two SOCAMM modules. Each STX box contains two BF-4 units, totaling two Vera CPUs, four CX-9 NICs, and four SOCAMM modules. For the whole STX rack, it has a total of 16 boxes, implying 32 Vera CPUs, 64 CX-9 NICs, and 64 SOCAMMs.

VR NVL72 裡的 BF-4 由一顆 Grace CPU + 一顆 CX-9 NIC 組成；STX reference design 的 BF-4 則包含一顆 Vera CPU、兩顆 CX-9 NIC、兩個 SOCAMM module。每個 STX box 放兩組 BF-4，所以每 box 合計兩顆 Vera、四顆 CX-9、四個 SOCAMM。整個 STX rack 有 16 個 box，也就是總共 32 顆 Vera CPU、64 顆 CX-9 NIC、64 個 SOCAMM。

Atomic Claim 396/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0308::RESEG04

Claim: STX reference design 的 BF-4 包含 2 個 SOCAMM modules。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 397/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0316

Claim: 整個 STX rack 共有 64 個 SOCAMMs。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 398/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0309::SPLIT03

Claim: 每個 STX box 合計四顆 CX-9 NICs。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 399/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0308::RESEG03

Claim: STX reference design 的 BF-4 包含 2 顆 CX-9 NICs。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 400/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0309::SPLIT02

Claim: 每個 STX box 合計兩顆 Vera CPUs。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 401/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0313

Claim: 整個 STX rack 共有 16 個 boxes。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 402/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0310::SPLIT03

Claim: 整個 STX rack 共有 64 顆 CX-9 NICs。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 403/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0309::SPLIT01

Claim: 每個 STX box 含兩個 BF-4 units。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 404/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0308::RESEG02

Claim: STX reference design 的 BF-4 包含 1 顆 Vera CPU。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 405/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0307::RESEG01

Claim: VR NVL72 的 BF-4 包含 1 顆 Grace CPU。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 406/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::NIEK2-0314

Claim: 因此整個 STX rack 共有 32 顆 Vera CPUs。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 407/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0309::SPLIT04

Claim: 每個 STX box 合計四個 SOCAMM modules。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 408/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0308::RESEG01

Claim: VR NVL72 的 BF-4 包含 1 顆 CX-9 NIC。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

STX Rack (left). Source: Nvidia, SemiAnalysis

STX Rack（左）。Source: Nvidia, SemiAnalysis

The STX announcement included a typical Nvidia show of strength where they named all major storage vendors as supporting STX, including AIC, Cloudian, DDN, Dell Technologies, Everpure, Hitachi Vantara, HPE, IBM, MinIO, NetApp, Nutanix, Supermicro, Quanta Cloud Technology (QCT), VAST Data and WEKA.

STX announcement 又上演了一次典型 Nvidia『秀肌肉』：直接把幾乎所有主要 storage vendor 都列成 STX supporter，包括 AIC、Cloudian、DDN、Dell Technologies、Everpure、Hitachi Vantara、HPE、IBM、MinIO、NetApp、Nutanix、Supermicro、Quanta Cloud Technology（QCT）、VAST Data、WEKA。

Atomic Claim 409/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0312::SPLIT10

Claim: 在 STX announcement 中，Nvidia 將 NetApp 列為支援 STX 的 storage vendor。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 410/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0312::SPLIT07

Claim: 在 STX announcement 中，Nvidia 將 HPE 列為支援 STX 的 storage vendor。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 411/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0312::SPLIT08

Claim: 在 STX announcement 中，Nvidia 將 IBM 列為支援 STX 的 storage vendor。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 412/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0312::SPLIT01

Claim: 在 STX announcement 中，Nvidia 將 AIC 列為支援 STX 的 storage vendor。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 413/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0312::SPLIT12

Claim: 在 STX announcement 中，Nvidia 將 Supermicro 列為支援 STX 的 storage vendor。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 414/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0312::SPLIT15

Claim: 在 STX announcement 中，Nvidia 將 WEKA 列為支援 STX 的 storage vendor。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 415/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0312::SPLIT13

Claim: 在 STX announcement 中，Nvidia 將 Quanta Cloud Technology (QCT) 列為支援 STX 的 storage vendor。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 416/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0312::SPLIT09

Claim: 在 STX announcement 中，Nvidia 將 MinIO 列為支援 STX 的 storage vendor。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 417/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0312::SPLIT02

Claim: 在 STX announcement 中，Nvidia 將 Cloudian 列為支援 STX 的 storage vendor。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 418/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0312::SPLIT04

Claim: 在 STX announcement 中，Nvidia 將 Dell Technologies 列為支援 STX 的 storage vendor。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 419/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0312::SPLIT03

Claim: 在 STX announcement 中，Nvidia 將 DDN 列為支援 STX 的 storage vendor。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 420/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0312::SPLIT14

Claim: 在 STX announcement 中，Nvidia 將 VAST Data 列為支援 STX 的 storage vendor。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 421/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0312::SPLIT05

Claim: 在 STX announcement 中，Nvidia 將 Everpure 列為支援 STX 的 storage vendor。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 422/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0312::SPLIT11

Claim: 在 STX announcement 中，Nvidia 將 Nutanix 列為支援 STX 的 storage vendor。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 423/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0312::SPLIT06

Claim: 在 STX announcement 中，Nvidia 將 Hitachi Vantara 列為支援 STX 的 storage vendor。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Put together, BlueField-4, CMX, and STX represent NVIDIA’s broader effort to standardize how clusters are designed at the storage layer. NVIDIA has captured the compute and network layer, and is actively moving into the storage, software, and infrastructure operations layers over time.

合在一起看，BlueField-4、CMX、STX 代表 NVIDIA 正試圖把 cluster 在 storage layer 的設計方式也標準化。NVIDIA 已掌握 compute、network layer，現在正逐步往 storage、software、infrastructure operations layer 繼續延伸。

Atomic Claim 424/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0315::RESEG02

Claim: NVIDIA 正逐步進入 storage layer。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 425/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0315::RESEG03

Claim: NVIDIA 正逐步進入 software layer。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 426/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0314::RESEG01

Claim: SemiAnalysis 認為 NVIDIA 已經掌握 compute layer。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 427/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0313

Claim: 原文判斷：合併來看，BlueField-4、CMX 與 STX 代表 NVIDIA 更廣泛地嘗試標準化 cluster storage layer 設計。
Frame: NARY_RELATION · Mode: INFERRED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 428/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0315::RESEG01

Claim: SemiAnalysis 認為 NVIDIA 已經掌握 network layer。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 429/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0315::RESEG04

Claim: NVIDIA 正逐步進入 infrastructure operations layer。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Now behind the paywall, we will share some more details on how all of this impacts the supply chain. Including beneficiaries of the LPX system, and the updated Kyber racks. We will also reveal a rack concept that Nvidia has yet to announce.

接下來 paywall 後，我們會分享這些變化對 supply chain 的更多影響，包括 LPX system、更新版 Kyber rack 的受益者，也會揭露一個 Nvidia 尚未正式宣布的 rack concept。

Feynman NVL1152 Networking Topologies

Within each Feynman Kyber rack, we tentatively assume double the bandwidth per logical GPU and double the NVLink Switch bandwidth to 28.8T and 57.6T respectively. Though Jensen, in the Financial Q+A the day after the GTC Keynote, characterized NVL1152 as “all CPO”, the key technical blog outlining the new rack form factors ↗ only strictly referenced CPO for rack to rack interconnect. We will discuss the potential topography for both options.

對每個 Feynman Kyber rack，我們暫時假設每 logical GPU bandwidth 加倍、NVLink Switch bandwidth 也加倍，分別到 28.8T、57.6T。雖然 Jensen 在 GTC Keynote 隔天 Financial Q+A 把 NVL1152 描述成『all CPO』，但介紹新 rack form factor 的核心 technical blog ↗ 嚴格來說只明確提到 rack-to-rack interconnect 使用 CPO。後面我們會分別討論兩種情境可能的 topology。

Atomic Claim 430/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0321::SPLIT02

Claim: Nvidia 的 rack form-factor technical blog 僅明確提到 CPO 用於 NVL1152 的 rack-to-rack interconnect。
Frame: ATTRIBUTE · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 431/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0320::RESEG01

Claim: 對每個 Feynman Kyber rack，SemiAnalysis 暫時假設 NVLink Switch bandwidth 會翻倍至 57.6T。
Frame: CLAIM_ONLY · Mode: HYPOTHETICAL · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 432/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0321::SPLIT01

Claim: Jensen 在 GTC keynote 隔日的 Financial Q+A 將 NVL1152 描述為「all CPO」。
Frame: ATTRIBUTE · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 433/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0319::RESEG01

Claim: 對每個 Feynman Kyber rack，SemiAnalysis 暫時假設每個 logical GPU 的 bandwidth 會翻倍至 28.8T。
Frame: CLAIM_ONLY · Mode: HYPOTHETICAL · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

To double the scale-up bandwidth using copper interconnect, NVIDIA would have to achieve a per lane bandwidth of 448Gbit/s uni-di (and implemented with simulatanous bi-directional SerDes so that each physical channel carries 448G of RX and 448G of TX) . However, this is a challenging feat as they would first have to prove the feasibility of 448Gb/s PAM4 SerDes at large volumes, then implement echo cancellation to achieve bidirectional bandwidth ↗, which is in itself extremely difficult. We believe Nvidia is going for 448G uni-di only.

如果要靠 copper 把 scale-up bandwidth 再翻倍，NVIDIA 必須讓每 lane 達到 448Gbit/s uni-di；若還要 simultaneous bidirectional，等於每 physical channel 同時承載 448G RX + 448G TX。這非常困難，因為 Nvidia 不只要先證明 448Gb/s PAM4 SerDes 能大規模量產，還得再做 echo cancellation 才能達成 bi-directional bandwidth ↗，而這本身就極其困難。我們認為 Nvidia 實際上會走 448G uni-di。

Atomic Claim 434/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0325

Claim: SemiAnalysis 判斷 Nvidia 會採 448G uni-di only。
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 435/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0323

Claim: 在原文所述假設情境中，若要用 copper interconnect 將 scale-up bandwidth 翻倍，NVIDIA 必須做到每 lane 448Gbit/s uni-di；若採 simultaneous bi-directional，單一 physical channel 需同時承載 448G RX 與 448G TX。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 436/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0324::SPLIT01

Claim: 要走向 simultaneous bidirectional 448G，首先必須證明 448Gb/s PAM4 SerDes 能在 large volumes 下可行。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 437/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0324::SPLIT02

Claim: 之後還必須實作 echo cancellation 才能達成 bidirectional bandwidth，而這本身極具挑戰。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: SemiAnalysis

Feynman could use in-rack optics, where switch blades are blind-mated to the midplane using optical connectors and thin fiber strands can be used to connect the optical connectors to the NVLink 8 Switches in place of flyover cables., but we believe this is very unlikely.

Feynman 理論上也可以在 rack 內使用 optics，例如 switch blade 透過 optical connector blind-mate 到 midplane，再用細 fiber strand 取代 flyover cable，把 optical connector 接到 NVLink 8 Switch；但我們認為這種設計非常不可能。

Atomic Claim 438/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0326

Claim: Feynman 可能採用 in-rack optics，讓 switch blade 透過 optical connector blind-mate 到 midplane。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 439/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0328

Claim: SemiAnalysis considers the discussed Feynman in-rack optical-backplane option highly unlikely.
Frame: CLAIM_ONLY · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 440/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0327

Claim: 在 SemiAnalysis 認為非常不可能的 Feynman in-rack optics 情境中，細 fiber strands 可取代 flyover cables，連接 optical connectors 與 NVLink 8 switches。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: SemiAnalysis

For rack-to-rack interconnect, we explore two different topologies. The first is a two-layer CLOS network that is similar to the Oberon form factor, but with twice the bandwidth of each GPU and NVLink switch.

Rack-to-rack interconnect 方面，我們探索兩種 topology。第一種是類似 Oberon form factor 的 two-layer CLOS network，但每顆 GPU、NVLink switch bandwidth 都加倍。

Atomic Claim 441/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0330

Claim: SemiAnalysis 探討的第一種 Feynman rack-to-rack 方案是 two-layer CLOS network，類似 Oberon form factor，但每 GPU 與 NVLink switch bandwidth 加倍。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: COMPLETE
開啟本版 Claim 與證據

Source: SemiAnalysis

The second is a reconfigurable dragonfly topology using OCS switches to connect the 8 racks. The number of OCS ports required for this topology remains tentative.

第二種是 reconfigurable dragonfly topology，用 OCS switch 連接 8 個 rack；這種 topology 最後需要多少 OCS port，目前仍是 tentative。

Atomic Claim 442/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0332

Claim: The number of OCS ports for the hypothetical eight-rack Feynman reconfigurable dragonfly topology remains tentative.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 443/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0331

Claim: SemiAnalysis 探討的第二種 Feynman rack-to-rack 方案是以 OCS switches 連接八個 racks 的 reconfigurable dragonfly topology；OCS port 數仍未定。
Frame: CLAIM_ONLY · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟本版 Claim 與證據

Source: SemiAnalysis

GTC 2026 Supply Chain Implications

Here, we will discuss our findings on where we see there are big changes in content for the supply chain coming out of all these announcements at GTC.

下面會整理這次 GTC announcement 中，我們認為對 supply-chain content 變化最大的幾個地方。

AlphaWave 112G Serdes in LP30

It may surprise readers that Qualcomm has IP in the Groq LPU 3 chip! More specifically it is AlphaWave, which Qualcomm acquired last year, that is providing the 112G SerDes for Groq’s C2C. AlphaWave was selected as the only IP provider that has high speed SerDes for Samsung Foundry. It was AlphaWave’s SerDes that caused issues for Groq LPU 2. Alphawave will continue to be used for the LP35, but Nvidia will of course use their own NVLink SerDes IP from LP40 when it transitions back to TSMC.

讀者可能會很意外：Groq LPU 3 裡竟然有 Qualcomm IP。更精確地說，是 Qualcomm 去年收購的 AlphaWave，為 Groq C2C 提供 112G SerDes。AlphaWave 當初被選中，是因為它是少數在 Samsung Foundry 擁有 high-speed SerDes 的 IP provider。LPU 2 出問題的，也正是 AlphaWave SerDes。LP35 仍會繼續用 AlphaWave；但到 LP40 轉回 TSMC 後，Nvidia 當然會改用自己的 NVLink SerDes IP。

Atomic Claim 444/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0336

Claim: The article says AlphaWave was selected for Groq because it was the only IP provider offering the required high-speed SerDes for Samsung Foundry.
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 445/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0334

Claim: 值得注意的是，Qualcomm 的 IP 實際存在於 Groq LPU 3 晶片中。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 446/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0339

Claim: 原文預期：當 LP40 轉回 TSMC 後，Nvidia 將使用自家的 NVLink SerDes IP。
Frame: NARY_RELATION · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 447/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0335::SPLIT01

Claim: Qualcomm 去年收購了 AlphaWave。
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 448/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0338

Claim: 原文預期：LP35 仍會繼續使用 Alphawave。
Frame: ATTRIBUTE · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 449/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0335::SPLIT02

Claim: AlphaWave 為 Groq 的 C2C 提供 112G SerDes。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 450/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0337

Claim: AlphaWave 的 SerDes 曾造成 Groq LPU 2 的問題。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

LPX PCB

Next, we mentioned that a very high spec PCB is required for the LPX compute tray. We estimate that each compute tray main board PCB will carry $7k ASP. The suppliers for this are Victory Giant and WUS. Of course, there are several other PCB modules in the compute tray, but they do not need a high spec. Nvidia is continuing with their cable-less philosophy similar to the Vera Rubin compute tray which requires a lot of board-to-board connectors, which brings us to the next big beneficiary.

再來，前面提到 LPX compute tray 需要非常高規格 PCB。我們估計每個 compute tray main-board PCB ASP 約 $7k，supplier 是 Victory Giant、WUS。Compute tray 內當然還有其他 PCB module，但不需要同等級 specification。Nvidia 也延續 Vera Rubin compute tray 類似的 cableless philosophy，因此需要大量 board-to-board connector，這就帶到下一個主要受益者。

Atomic Claim 451/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0343

Claim: Other PCB modules in the LPX compute tray do not require the main board’s high specifications.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 452/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0341

Claim: SemiAnalysis estimates a US$7,000 ASP for each LPX compute-tray main-board PCB.
Frame: ATTRIBUTE · Mode: ESTIMATED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 453/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0342::SPLIT02

Claim: WUS is identified as a supplier of the LPX compute-tray main-board PCB.
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 454/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0344

Claim: Nvidia 持續採用類似 Vera Rubin compute tray 的 cable-less philosophy，因此需要大量 board-to-board connectors。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 455/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0342::SPLIT01

Claim: Victory Giant is identified as a supplier of the LPX compute-tray main-board PCB.
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 456/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0340

Claim: LPX compute tray 需要非常高規格的 PCB。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Cables and Connectors: Amphenol Continues to Benefit

For the LPX, Amphenol will be a beneficiary for all the connectors for the backplane. Each LPX node requires 16 80DP Paladin connectors for the backplane. There are also board to board connectors required to connect all the various modules within the tray: the main LPU board with the host CPU module and the OSFP/QSFP modules that sit below the CPU module, the front-end NIC module, and the management module. Amphenol will supply the cable backplane too which is 8,160 DP per rack.

LPX 方面，Amphenol 會受益於 backplane 與 tray 內大量 connector。每個 LPX node 需要 16 個 80DP Paladin connector 連 backplane。此外，tray 內各 module 互連也需要 board-to-board connector：main LPU board 要連 host CPU module、位於 CPU module 下方的 OSFP/QSFP module、frontend NIC module、management module。Amphenol 也會供應 cable backplane，整 rack 規模達 8,160 DP。

Atomic Claim 457/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0345

Claim: 原文預期：在 LPX 中，Amphenol 會受益於 backplane 所需的 connector content。
Frame: NARY_RELATION · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 458/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0348::RESEG02

Claim: LPX tray 的 OSFP／QSFP modules 位於 host CPU module 下方。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 459/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0349

Claim: Amphenol 也會供應 cable backplane，每 rack 為 8,160 DP。
Frame: RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 460/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0348::RESEG01

Claim: LPX tray 內需要 board-to-board connectors 連接主 LPU board 與 OSFP／QSFP modules。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 461/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0346

Claim: 每個 LPX node 的 backplane 需要 16 個 80DP Paladin connectors。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 462/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0348::RESEG04

Claim: LPX tray 內需要 board-to-board connectors 連接主 LPU board 與 management module。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 463/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0348::RESEG03

Claim: LPX tray 內需要 board-to-board connectors 連接主 LPU board 與 front-end NIC module。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 464/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0347::RESEG01

Claim: LPX tray 內需要 board-to-board connectors 連接主 LPU board 與 host CPU module。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

NVL288 System

For the Vera Rubin Ultra NVL288 System that we discussed above, we would say the cable backplane return for Kyber. If Rubin Ultra is deployed in such a form factor – each of the Rubin Ultra GPUs will have a scale-up bandwidth of 14.4Tbit/s uni-di, requiring 144 DPs of cables to connect to the NVSwitches. 144 DPs times 288 GPUs means a total of 41,472 DPs to connect this larger world size domain. This is a lot of cables, so it is more of an upper bound of how much cable content could be used here. If there is oversubscription or if the inter-rack connection is made through the switches – it is possible fewer DPs would be needed.

前面討論的 Vera Rubin Ultra NVL288，如果真的採這個 form factor，我們認為 Kyber 會重新需要大量 cable backplane。每顆 Rubin Ultra GPU scale-up bandwidth 14.4Tbit/s uni-di，需要 144 DP cable 連 NVSwitch；144 DP × 288 GPU = 41,472 DP，才能連完整個 larger world-size domain。這是非常巨量的 cable，因此比較像 cable content 的 upper-bound case。若 architecture 有 oversubscription，或 rack-to-rack connection 改經 switch，實際所需 DP 可能更少。

Atomic Claim 465/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0354

Claim: In the Rubin Ultra NVL288 concept, oversubscription or switched inter-rack connections could reduce the required DP count.
Frame: CLAIM_ONLY · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 466/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0350

Claim: 在原文所述假設情境中，對前述 Vera Rubin Ultra NVL288 system，SemiAnalysis 認為 Kyber 會重新採用 cable backplane。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 467/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0353

Claim: In the discussed Rubin Ultra NVL288 concept, 41,472 total cable DPs is an upper-bound estimate for cable content.
Frame: CLAIM_ONLY · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 468/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0351

Claim: 若 Rubin Ultra 以此 form factor 部署，每顆 Rubin Ultra GPU 將提供 14.4Tbit/s uni-di scale-up bandwidth，並需要 144 DP cable 連到 NVSwitch。
Frame: NARY_RELATION · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 469/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0352

Claim: 若 Rubin Ultra NVL288 採每 GPU 144 DPs 的配置，144 × 288 = 41,472 DPs 是該連線情境的 cable-content 上界；oversubscription 或經 switches 跨 rack 可降低需求。
Frame: ATTRIBUTE · Mode: HYPOTHETICAL · Mapping: PARTIAL
開啟本版 Claim 與證據

FIT Joining the Party

Backplane cable cartridge and Paladin connector demand is so strong that Amphenol cannot keep up with supply. Amphenol has now completed licensing of the VR NVL72 backplane cable cartridge as well as Paladin HD connectors to FIT, who can now manufacture these components. This has been in the works for a long time but is finally settled. Amphenol will earn licensing fees from FIT’s sales of these licensed components.

Backplane cable cartridge、Paladin connector demand 強到 Amphenol 自己已供不應求。Amphenol 現在已完成把 VR NVL72 backplane cable cartridge、Paladin HD connector 授權給 FIT，FIT 可以自行製造。這件事談了很久，現在終於確定；Amphenol 會從 FIT 銷售這些 licensed component 中收取 licensing fee。

Atomic Claim 470/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0356::SPLIT01

Claim: Amphenol has completed licensing the VR NVL72 backplane cable cartridge to FIT, enabling FIT to manufacture it.
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 471/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0356::SPLIT02

Claim: Amphenol has completed licensing Paladin HD connectors to FIT, enabling FIT to manufacture them.
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 472/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0358

Claim: 原文預期：Amphenol 將從 FIT 銷售這些授權元件中取得 licensing fee。
Frame: NARY_RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 473/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0355

Claim: Backplane cable cartridge 與 Paladin connector demand 強到 Amphenol 無法完全供應。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Kyber Voronoi – Another FIT Win?

The Kyber midplane will utilize many 8×19 DP connectors to interface with the compute trays at the front of the rack, and to the switch blades in the back of the rack.

Kyber midplane 會使用大量 8×19 DP connector，前側連 compute tray，後側連 switch blade。

Atomic Claim 474/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0359

Claim: Kyber midplane 將使用大量 8×19 DP connector，連接 rack 前方的 compute tray 與後方 switch blade。
Frame: NARY_RELATION · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

For Kyber, Nvidia is now in the driver’s seat when it comes to IP and they have designed a proprietary connector spec named Voronoi, so it will no longer be the Amphenol Paladin connector. There are three vendors bidding for the project: FIT, Molex and Amphenol. FIT appears to be leading the market for these connectors, but Amphenol is reportedly also working together closely with FIT to manufacture the connectors. The design and implementation of Voronoi remains in flux, but both FIT and Amphenol will need to ramp significant production volume with the specification licensed from Nvidia.

Kyber connector IP 方面，Nvidia 現在已掌握主導權，並設計一套 proprietary spec，名稱叫 Voronoi，因此不再使用 Amphenol Paladin connector。目前有三家 vendor 競標：FIT、Molex、Amphenol。FIT 看起來暫時領先，但據了解 Amphenol 也和 FIT 密切合作，共同生產 connector。Voronoi design、implementation 仍在變動，但 FIT、Amphenol 都必須依 Nvidia 授權的 specification，準備 ramp 非常大的 production volume。

Atomic Claim 475/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0365

Claim: Voronoi 的設計與實作目前仍在變動中。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 476/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0362::SPLIT01

Claim: FIT is bidding for the Nvidia Kyber Voronoi connector project.
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 477/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0362::SPLIT02

Claim: Molex is bidding for the Nvidia Kyber Voronoi connector project.
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 478/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0362::SPLIT03

Claim: Amphenol is bidding for the Nvidia Kyber Voronoi connector project.
Frame: RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 479/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0364

Claim: Amphenol is reportedly working closely with FIT to manufacture Kyber Voronoi connectors.
Frame: NARY_RELATION · Mode: ATTRIBUTED · Mapping: PARTIAL
開啟本版 Claim 與證據

Atomic Claim 480/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0363

Claim: SemiAnalysis judges FIT to be leading the competition for Nvidia Kyber Voronoi connectors.
Frame: ATTRIBUTE · Mode: INFERRED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 481/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0366::SPLIT01

Claim: FIT will need to ramp significant Voronoi connector production using the specification licensed from Nvidia.
Frame: RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 482/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0366::SPLIT02

Claim: Amphenol will need to ramp significant Voronoi connector production using the specification licensed from Nvidia.
Frame: RELATION · Mode: EXPECTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 483/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0361

Claim: Nvidia 設計了名為 Voronoi 的 proprietary connector spec，因此不再沿用 Amphenol 的 Paladin connector。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

The midplane, switch tray and compute tray all feature female side connectors which will require the use of a spring-loaded male part that protect the pins and interface between both sides. The density of these connectors will ultimately be much higher than Amphenol’s Paladin connectors.

Midplane、switch tray、compute tray 都使用 female-side connector，因此需要一個 spring-loaded male component 介於兩側，保護 pin 並完成 interface。這些 connector 最後的 density 會明顯高於 Amphenol Paladin。

Atomic Claim 484/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0367::SPLIT03

Claim: The Kyber compute tray uses female-side connectors.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 485/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0367::SPLIT02

Claim: The Kyber switch tray uses female-side connectors.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 486/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0367::SPLIT04

Claim: The female-side Kyber connectors require a spring-loaded male part to protect the pins and interface between the two sides.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 487/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0367::SPLIT01

Claim: The Kyber midplane uses female-side connectors.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 488/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0368

Claim: 原文預期 Kyber 新 connector 的最終密度將遠高於 Amphenol Paladin connectors。
Frame: COMPARISON · Mode: EXPECTED · Mapping: PARTIAL
開啟本版 Claim 與證據

More is available to our institutional subscribers sales@semianalysis.com.

更多內容提供給 institutional subscriber，洽詢 sales@semianalysis.com。

Mid-board Optics – Nvidia’s War on Pluggables

Interestingly, the Kyber rack exhibited at the GTC 2026 show floor is missing OSFP cages for scale-out networking. Instead, we only see 4x MPO ports from each compute tray. This design has effectively taken key pluggable transceiver items (driver, TIA, etc.) other than the DSP and put them on a Midboard Optical Module (MBOM) which then connects to the PCB via a land grid array (LGA) socket. Two CX-9s share one MBOM, which then connects to the MPO faceplate via a short fiber connection. The MBOM provides two MPO ports at 2x800G each for 1.6T of total connectivity.

有趣的是，GTC 2026 show floor 展示的 Kyber rack 沒有 scale-out networking 用的 OSFP cage；每個 compute tray 前面看到的反而只有 4 個 MPO port。這套 design 等於把 pluggable transceiver 的主要元件——driver、TIA 等，DSP 除外——移到 Midboard Optical Module（MBOM），再透過 land-grid-array（LGA）socket 接到 PCB。兩顆 CX-9 共用一個 MBOM，之後用短 fiber connection 接到 MPO faceplate。每個 MBOM 提供兩個 MPO port，每 port 2×800G，總 connectivity 1.6T。

Atomic Claim 489/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0372::RESEG02

Claim: Kyber 的 mid-board optics design 將 pluggable transceiver 的 TIA 移到 Midboard Optical Module（MBOM）上。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 490/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0372::RESEG01

Claim: Kyber 的 mid-board optics design 將 pluggable transceiver 的 driver 移到 Midboard Optical Module（MBOM）上。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 491/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0374::SPLIT01

Claim: In the Kyber design exhibited at GTC 2026, two CX-9 NICs share one MBOM.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 492/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0374::SPLIT02

Claim: In the Kyber design exhibited at GTC 2026, the MBOM connects to the MPO faceplate via a short fiber connection.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 493/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0372::RESEG03

Claim: Kyber 的 mid-board optics design 不把 DSP 移到 Midboard Optical Module（MBOM）上。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 494/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-MBOM-PORTS

Claim: The GTC 2026 Kyber MBOM is described as providing two MPO ports.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 495/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0370

Claim: GTC 2026 展場展示的 Kyber rack 缺少供 scale-out networking 使用的 OSFP cages。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 496/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0373::RESEG01

Claim: Midboard Optical Module（MBOM）透過 land grid array socket 連接到 PCB。
Frame: CLAIM_ONLY · Mode: ASSERTED · Mapping: CLAIM_ONLY
開啟本版 Claim 與證據

Atomic Claim 497/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA-COV-SA-MBOM-TOTAL

Claim: The article states total connectivity of 1.6T for the GTC 2026 Kyber MBOM; the same sentence has inconsistent per-port wording.
Frame: ATTRIBUTE · Mode: ATTRIBUTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 498/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0371

Claim: The Kyber compute tray displayed at GTC 2026 shows four MPO ports instead of scale-out OSFP cages.
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

4 MPO ports on the left, rather than OSFP cages. Source: Nvidia, SemiAnalysis

左側 4 個 MPO port，而不是 OSFP cage。Source: Nvidia, SemiAnalysis

The use of MBOM would block the use of any form of pluggable transceiver or AEC, and naturally hyperscalers are saying “CP-Hell No” to that idea and are continuing to push for an OSFP cage so they can continue using pluggables.

但使用 MBOM 等於直接封死 pluggable transceiver、AEC 的可能性，hyperscaler 當然對此說『CP-Hell No』，仍持續推動保留 OSFP cage，讓它們可以繼續使用 pluggable。

Atomic Claim 499/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0378

Claim: In the displayed Kyber MBOM design, MBOM would prevent the use of pluggable transceivers or AECs.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 500/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0379

Claim: Hyperscaler 對 MBOM 方案明顯反對，並持續要求保留 OSFP cage，以便繼續使用 pluggable。
Frame: NARY_RELATION · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

It is important to point out that many aspects of the Kyber design are still in flux and there could still be a number of design changes before Kyber racks are actually deployed. After all – the change from the four canister design to a two compute tray canister + one switch blade bank is already a huge change.

需要強調，Kyber design 很多部分仍在變，真正 deployment 前還可能有不少 design change。光是從原本四個 canister，改成兩個 compute-tray canister + 一個 switch-blade bank，就已經是非常大的變動。

Atomic Claim 501/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0380

Claim: 必須注意，Kyber design 的許多部分目前仍在變動中。
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 502/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0382::REPAIR01

Claim: The described Kyber prototype changes from four canisters to two compute-tray canisters plus one switch-blade bank; the design remains in flux before deployment.
Frame: ATTRIBUTE · Mode: ASSERTED · Mapping: COMPLETE
開啟本版 Claim 與證據

Atomic Claim 503/503 · 2026-03-24_nvidia-the-inference-kingdom-expands::SA260324R-0381

Claim: 在 Kyber rack 真正部署前，仍可能出現多項 design change。
Frame: ATTRIBUTE · Mode: HYPOTHETICAL · Mapping: COMPLETE
開啟本版 Claim 與證據

Research Vault

探索

2026-03-24_nvidia-the-inference-kingdom-expands

SA Article Coverage Review · 2026-03-24_nvidia-the-inference-kingdom-expands

Coverage Summary

Missing Claim Notes

SA Full Text + Translation + Atomic Claims

Nvidia – The Inference Kingdom Expands

Groq

LPU chip

SRAM and Memory Hierarchy

GPU and LPU Integration: Attention FFN Disaggregation (AFD)

Speculative Decoding

LPX Rack System

LPX Compute Tray

LPU Network

Intra-Tray Topology

Inter-node/Intra-rack

Inter-rack

Nvidia’s CPO Roadmap

Oberon and Kyber Updates, Larger World Sizes Introduced, More Networking Updates

Rubin Ultra NVL288

Rubin Ultra NVL576

Feynman

CMX and STX

CMX

STX

Feynman NVL1152 Networking Topologies

GTC 2026 Supply Chain Implications

AlphaWave 112G Serdes in LP30

LPX PCB

Cables and Connectors: Amphenol Continues to Benefit

NVL288 System

FIT Joining the Party

Kyber Voronoi – Another FIT Win?

Mid-board Optics – Nvidia’s War on Pluggables

關係圖譜

目錄

反向連結