clm_d9f25137d0f4499886a3131ca6db76e8_r1

IX2-0406

① SA Source

• Source: 開啟完整 SA 文章
• Section: WideEP
• Line hint: 640

Context Before

The forward pass proceeds in two phases per layer. During attention, each GPU acts as an independent data-parallel rank, processing its own subset of requests using its replicated non-expert weights, no inter-GPU communication is needed. During the MoE phase, a lightweight router determines which experts each token requires, and tokens are dispatched to the appropriate GPUs via all-to-all communication. Each GPU executes its local experts on only the tokens routed to it, and results are returned via a second all-to-all.

Evidence

All 256 experts per layer are divided evenly among the 8 GPUs, whereas attention along with other non-expert weights (shared expert, gating network, RMSNorm, LM head, etc.)

Context After

The obvious way to scale is replication: deploy N independent EP8 instances across N nodes. Each instance serves requests independently with no cross-node communication. This scales throughput linearly, but each GPU still holds 32 experts per layer, and each token activates at most 8 of those 32 local experts. 75% of expert weights sit cold in HBM.

Wide expert parallelism (WideEP) takes a different approach by scaling EP _across _nodes rather than replicating independent instances. On a 64-GPU cluster (8 nodes), DP64/EP64 places only 256/64 = 4 experts per layer per GPU, each still holding a full replica of the non-expert weights. During the MoE phase, tokens from all 64 DP ranks are dispatched via all-to-all to the GPUs hosting their routed experts.

② Atomic Claim

每層 256 個 experts 平均分配到 8 顆 GPUs；attention 與其他 non-expert weights（shared expert、gating network、RMSNorm、LM head 等）則採複製方式。

• Epistemic Mode: ASSERTED
• Mapping Status: COMPLETE

③ Semantic Frame

{
  "attribute": "COUNT",
  "context_nodes": [],
  "entity": {
    "id": "04_knowledge_base/GPU",
    "label": "GPUs"
  },
  "frame_type": "ATTRIBUTE",
  "qualifiers": {
    "condition_text": null,
    "numeric_mentions": [
      "256",
      "8"
    ],
    "temporal_mentions": []
  },
  "value": {
    "numeric_mentions": [
      "256",
      "8"
    ],
    "value_text": "每層 256 個 experts 平均分配到 8 顆 GPUs；attention 與其他 non-expert weights（shared expert、gating network、RMSNorm、LM head 等）則採複製方式。"
  }
}

④ Canonical Entity Mapping

Role	Surface Label	Canonical Target
entity	GPUs	GPU

⑤ Human Review

請在 Properties 逐項確認：

• 原文 → Atomic Claim 是否忠實
• Atomic Claim → Semantic Frame 是否忠實
• Canonical Entity mapping 是否正確
• Epistemic mode 是否保留原文語氣
• 最後選擇 review_action

Review state

Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。