IX2-0437
① SA Source
- Source: 開啟完整 SA 文章
- Section:
WideEP - Line hint:
658
Context Before
Adding tensor parallelism within groups of GPUs addresses this. In an EP64/DP8/TP8 configuration, the 64 GPUs are organized into 8 DP groups of 8 GPUs each. Within each TP group, the attention projections, shared expert, normalization, and LM head are sharded 8 ways, so each GPU holds only 1/8th of the non-expert weights. Across the full cluster, the 256 experts are still distributed one-per-4-GPUs as before.
Pure DEP has a single communication pattern: all-to-all for expert routing. Adding TP introduces a second all-reduce within each TP group for the attention and non-expert computations. The key design principle is to place TP groups within a single node, where NVLink or MNNVL provides high-bandwidth interconnect, and run EP/DP across nodes, where the all-to-all communication pattern can tolerate higher latency.
Evidence
Context After
Disaggregated Prefill
Disaggregated prefill, sometimes referred to as prefill-decode (PD) disaggregation, is the process of performing prefill and decode phases of LLM inference on separate nodes. Prefill occurs when a request is first processed, and a forward pass is computed on all tokens at once, thereby “prefilling” the KV cache for this request. This is a compute-intensive operation as all tokens feed through the forward pass in parallel. Tokens are then generated or “decoded” one at a time, loading the KV cache from HBM at each decode step. This is a memory-intensive process as the growing KV cache is constantly being loaded.
② Atomic Claim
在一個 group 內使用 TP=8，代表 8 顆 GPUs 必須共用同一 batch，且每個 decode step 都要同步，使有效 DP degree 從 64 降到 8。
- Epistemic Mode:
ASSERTED - Mapping Status:
COMPLETE
③ Semantic Frame
{
"additional_nodes": [],
"frame_type": "RELATION",
"object": {
"id": "04_knowledge_base/Decode",
"label": "decode"
},
"predicate": "USES",
"qualifiers": {
"condition_text": null,
"numeric_mentions": [
"8",
"64"
],
"temporal_mentions": []
},
"subject": {
"id": "04_knowledge_base/GPU",
"label": "GPUs"
}
}④ Canonical Entity Mapping
⑤ Human Review
請在 Properties 逐項確認：
- 原文 → Atomic Claim 是否忠實
- Atomic Claim → Semantic Frame 是否忠實
- Canonical Entity mapping 是否正確
- Epistemic mode 是否保留原文語氣
- 最後選擇
review_action
Review state
Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。