clm_8f6cb1fe77774e0397dba375ed85698f_r1

IX2-0440

① SA Source

• Source: 開啟完整 SA 文章
• Section: WideEP
• Line hint: 658

Context Before

Adding tensor parallelism within groups of GPUs addresses this. In an EP64/DP8/TP8 configuration, the 64 GPUs are organized into 8 DP groups of 8 GPUs each. Within each TP group, the attention projections, shared expert, normalization, and LM head are sharded 8 ways, so each GPU holds only 1/8th of the non-expert weights. Across the full cluster, the 256 experts are still distributed one-per-4-GPUs as before.

Pure DEP has a single communication pattern: all-to-all for expert routing. Adding TP introduces a second all-reduce within each TP group for the attention and non-expert computations. The key design principle is to place TP groups within a single node, where NVLink or MNNVL provides high-bandwidth interconnect, and run EP/DP across nodes, where the all-to-all communication pattern can tolerate higher latency.

Evidence

Per-token latency drops while peak concurrency also drops, sliding the configuration along the latency-throughput Pareto frontier relative to pure DEP

Context After

Disaggregated Prefill

Disaggregated prefill, sometimes referred to as prefill-decode (PD) disaggregation, is the process of performing prefill and decode phases of LLM inference on separate nodes. Prefill occurs when a request is first processed, and a forward pass is computed on all tokens at once, thereby “prefilling” the KV cache for this request. This is a compute-intensive operation as all tokens feed through the forward pass in parallel. Tokens are then generated or “decoded” one at a time, loading the KV cache from HBM at each decode step. This is a memory-intensive process as the growing KV cache is constantly being loaded.

② Atomic Claim

關於 EP：因此 per-token latency 下降，但 peak concurrency 也下降，configuration 會沿著 latency-throughput Pareto frontier 相對 pure DEP 移動。

• Epistemic Mode: ASSERTED
• Mapping Status: COMPLETE

③ Semantic Frame

{
  "attribute": "THROUGHPUT",
  "context_nodes": [],
  "entity": {
    "id": "04_knowledge_base/Expert Parallelism",
    "label": "EP"
  },
  "frame_type": "ATTRIBUTE",
  "qualifiers": {
    "condition_text": null,
    "numeric_mentions": [],
    "temporal_mentions": []
  },
  "value": {
    "numeric_mentions": [],
    "value_text": "關於 EP：因此 per-token latency 下降，但 peak concurrency 也下降，configuration 會沿著 latency-throughput Pareto frontier 相對 pure DEP 移動。"
  }
}

④ Canonical Entity Mapping

Role	Surface Label	Canonical Target
entity	EP	04_knowledge_base/Expert Parallelism

⑤ Human Review

請在 Properties 逐項確認：

• 原文 → Atomic Claim 是否忠實
• Atomic Claim → Semantic Frame 是否忠實
• Canonical Entity mapping 是否正確
• Epistemic mode 是否保留原文語氣
• 最後選擇 review_action

Review state

Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。