clm_c6136df890394f57a48173ce08821131_r1

IX2-0234

① SA Source

• Source: 開啟完整 SA 文章
• Section: Nvidia Disagg Prefill and WideEP
• Line hint: 388

Context Before

Generally, TP is preferred at lower concurrencies due to load balancing. At small batch sizes, EP suffers from uneven token-to-expert routing, leaving some GPUs underutilized while others are overloaded. TP avoids this since each GPU holds a slice of every expert and always gets an equal share of work. At lower concurrency, the cost of this load imbalance outweighs TP’s additional communication overhead.

At higher concurrencies, this tradeoff changes. Expert activation becomes more evenly distributed across larger batch sizes, and EP’s communication and weight-loading advantages dominate over TP’s expensive per-layer all-reduce. In the middle of the curve, hybrid TP+EP configurations balance both concerns using small TP groups within each expert for load balancing while EP is used across the wider set of GPUs to amortize weights and reduce communication.

Evidence

B300 disagg over IB has the same performance as GB300 with NVL72, since the workload is latency-bound, not bandwidth-bound

Context After

Prefill/decode disaggregation also plays a role. Prefill is compute-heavy and bursty; decode is memory-bandwidth-bound and steady-state. When they share the same GPUs, they interfere with each other, causing latency jitter and wasted capacity. Separating them onto dedicated GPU pools lets each run a workload matched to its characteristics, improving effective utilization. This is why disaggregated B200 configs outperform single-node B200 in the middle of the throughput-interactivity curve. PD separation combined with wider EP across more GPUs over IB amortizes weights more efficiently than cramming both phases onto a single 8-GPU node.

Side Note: the 10x inference engineers at TogetherAI noticed an pattern for multi-turn traffic where the requirements of first turn prefill is much different from the following turns prefill’s and disaggregrated it leading to better TTFT performance. ↗

② Atomic Claim

B300 透過 IB 執行 disagg 的效能與採 NVL72 的 GB300 相同，因為 workload 受 latency 限制，而不是 bandwidth 限制。

• Epistemic Mode: ASSERTED
• Mapping Status: COMPLETE

③ Semantic Frame

{
  "comparison_expression": "B300 透過 IB 執行 disagg 的效能與採 NVL72 的 GB300 相同，因為 workload 受 latency 限制，而不是 bandwidth 限制。",
  "entities": [
    {
      "id": "04_knowledge_base/NVIDIA B300",
      "label": "B300"
    },
    {
      "id": "04_knowledge_base/GB300",
      "label": "GB300"
    }
  ],
  "frame_type": "COMPARISON",
  "metric": "BANDWIDTH",
  "operator": "EQUAL_TO",
  "qualifiers": {
    "condition_text": null,
    "numeric_mentions": [],
    "temporal_mentions": []
  }
}

④ Canonical Entity Mapping

Role	Surface Label	Canonical Target
comparison_entity_0	B300	04_knowledge_base/NVIDIA B300
comparison_entity_1	GB300	GB300

⑤ Human Review

請在 Properties 逐項確認：

• 原文 → Atomic Claim 是否忠實
• Atomic Claim → Semantic Frame 是否忠實
• Canonical Entity mapping 是否正確
• Epistemic mode 是否保留原文語氣
• 最後選擇 review_action

Review state

Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。