clm_db28f4e9e0274c8e9c6d625f7099dccd_r1

IX2-0428

① SA Source

• Source: 開啟完整 SA 文章
• Section: WideEP
• Line hint: 654

Context Before

_A WideEP EP64 DP64 deployment of DeepSeek R1. All 256 experts per layer are divided evenly among the 64 GPUs (8 nodes), and attention and other non-expert weights (shared expert, gating network, RMSNorm, LM head, etc.) are replicated across all 64 DP ranks. _Source: SemiAnalysis

The above configurations use only DP+EP (also known as DEP), where each GPU holds a full replica of all non-expert weights. As GPU count grows, this replication becomes increasingly wasteful. On a 64-GPU DP64/EP64 deployment, every GPU stores an identical copy of the ~40B non-expert parameters.

Evidence

In an EP64/DP8/TP8 configuration, the 64 GPUs are organized into 8 DP groups of 8 GPUs each

Context After

Pure DEP has a single communication pattern: all-to-all for expert routing. Adding TP introduces a second all-reduce within each TP group for the attention and non-expert computations. The key design principle is to place TP groups within a single node, where NVLink or MNNVL provides high-bandwidth interconnect, and run EP/DP across nodes, where the all-to-all communication pattern can tolerate higher latency.

As always, the tradeoff is that of throughput versus latency. TP=8 within a group means those 8 GPUs now share a batch and must synchronize every decode step, reducing effective DP degree from 64 to 8. Per-GPU batching independence on the attention side is lost. But each DP group now processes attention 8x faster per step, since the matmul is split 8 ways across the TP group. Per-token latency drops while peak concurrency also drops, sliding the configuration along the latency-throughput Pareto frontier relative to pure DEP.

② Atomic Claim

在 EP64/DP8/TP8 configuration 中，64 顆 GPUs 被分成 8 個 DP groups，每組 8 顆 GPUs。

• Epistemic Mode: ASSERTED
• Mapping Status: COMPLETE

③ Semantic Frame

{
  "attribute": "COUNT",
  "context_nodes": [],
  "entity": {
    "id": "04_knowledge_base/GPU",
    "label": "GPUs"
  },
  "frame_type": "ATTRIBUTE",
  "qualifiers": {
    "condition_text": null,
    "numeric_mentions": [
      "64",
      "8"
    ],
    "temporal_mentions": []
  },
  "value": {
    "numeric_mentions": [
      "64",
      "8"
    ],
    "value_text": "在 EP64/DP8/TP8 configuration 中，64 顆 GPUs 被分成 8 個 DP groups，每組 8 顆 GPUs。"
  }
}

④ Canonical Entity Mapping

Role	Surface Label	Canonical Target
entity	GPUs	GPU

⑤ Human Review

請在 Properties 逐項確認：

• 原文 → Atomic Claim 是否忠實
• Atomic Claim → Semantic Frame 是否忠實
• Canonical Entity mapping 是否正確
• Epistemic mode 是否保留原文語氣
• 最後選擇 review_action

Review state

Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。