clm_fb29da21f0e74c828c0d5d94f90b594c_r1

IX2-0417

① SA Source

• Source: 開啟完整 SA 文章
• Section: WideEP
• Line hint: 646

Context Before

The obvious way to scale is replication: deploy N independent EP8 instances across N nodes. Each instance serves requests independently with no cross-node communication. This scales throughput linearly, but each GPU still holds 32 experts per layer, and each token activates at most 8 of those 32 local experts. 75% of expert weights sit cold in HBM.

Wide expert parallelism (WideEP) takes a different approach by scaling EP _across _nodes rather than replicating independent instances. On a 64-GPU cluster (8 nodes), DP64/EP64 places only 256/64 = 4 experts per layer per GPU, each still holding a full replica of the non-expert weights. During the MoE phase, tokens from all 64 DP ranks are dispatched via all-to-all to the GPUs hosting their routed experts.

Evidence

Second, 64 DP ranks funneling tokens through fewer experts per GPU increases tokens-per-expert, raising arithmetic intensity (more FLOPs per byte of weights loaded) and improving compute utilization

Context After

_A WideEP EP64 DP64 deployment of DeepSeek R1. All 256 experts per layer are divided evenly among the 64 GPUs (8 nodes), and attention and other non-expert weights (shared expert, gating network, RMSNorm, LM head, etc.) are replicated across all 64 DP ranks. _Source: SemiAnalysis

② Atomic Claim

第二，64 個 DP ranks 的 tokens 被集中到每顆 GPU 較少的 experts 上，使每個 expert 接收到更多 tokens，提高 arithmetic intensity，也就是每載入一 byte weights 可執行更多 FLOPs，進而提升 compute utilization。

• Epistemic Mode: ASSERTED
• Mapping Status: COMPLETE

③ Semantic Frame

{
  "comparison_expression": "第二，64 個 DP ranks 的 tokens 被集中到每顆 GPU 較少的 experts 上，使每個 expert 接收到更多 tokens，提高 arithmetic intensity，也就是每載入一 byte weights 可執行更多 FLOPs，進而提升 compute utilization。",
  "entities": [
    {
      "id": "04_knowledge_base/GPU",
      "label": "GPU"
    },
    {
      "id": "04_knowledge_base/Arithmetic intensity",
      "label": "arithmetic intensity"
    }
  ],
  "frame_type": "COMPARISON",
  "metric": "COMPUTE_PERFORMANCE",
  "operator": "UNSPECIFIED_COMPARISON",
  "qualifiers": {
    "condition_text": null,
    "numeric_mentions": [
      "64"
    ],
    "temporal_mentions": []
  }
}

④ Canonical Entity Mapping

Role	Surface Label	Canonical Target
comparison_entity_0	GPU	GPU
comparison_entity_1	arithmetic intensity	04_knowledge_base/Arithmetic intensity

⑤ Human Review

請在 Properties 逐項確認：

• 原文 → Atomic Claim 是否忠實
• Atomic Claim → Semantic Frame 是否忠實
• Canonical Entity mapping 是否正確
• Epistemic mode 是否保留原文語氣
• 最後選擇 review_action

Review state

Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。