IX2-0435
① SA Source
- Source: 開啟完整 SA 文章
- Section:
WideEP - Line hint:
656
Context Before
The above configurations use only DP+EP (also known as DEP), where each GPU holds a full replica of all non-expert weights. As GPU count grows, this replication becomes increasingly wasteful. On a 64-GPU DP64/EP64 deployment, every GPU stores an identical copy of the ~40B non-expert parameters.
Adding tensor parallelism within groups of GPUs addresses this. In an EP64/DP8/TP8 configuration, the 64 GPUs are organized into 8 DP groups of 8 GPUs each. Within each TP group, the attention projections, shared expert, normalization, and LM head are sharded 8 ways, so each GPU holds only 1/8th of the non-expert weights. Across the full cluster, the 256 experts are still distributed one-per-4-GPUs as before.
Evidence
run EP/DP across nodes, where the all-to-all communication pattern can tolerate higher latency
Context After
As always, the tradeoff is that of throughput versus latency. TP=8 within a group means those 8 GPUs now share a batch and must synchronize every decode step, reducing effective DP degree from 64 to 8. Per-GPU batching independence on the attention side is lost. But each DP group now processes attention 8x faster per step, since the matmul is split 8 ways across the TP group. Per-token latency drops while peak concurrency also drops, sliding the configuration along the latency-throughput Pareto frontier relative to pure DEP.
Disaggregated Prefill
② Atomic Claim
EP／DP 則跨 nodes 執行，因為 all-to-all communication pattern 對較高 latency 的容忍度更高。
- Epistemic Mode:
ASSERTED - Mapping Status:
PARTIAL
③ Semantic Frame
{
"comparison_expression": "EP／DP 則跨 nodes 執行，因為 all-to-all communication pattern 對較高 latency 的容忍度更高。",
"entities": [
{
"id": "04_knowledge_base/Expert Parallelism",
"label": "EP"
}
],
"frame_type": "COMPARISON",
"metric": "LATENCY",
"operator": "GREATER_THAN",
"qualifiers": {
"condition_text": null,
"numeric_mentions": [],
"temporal_mentions": []
}
}④ Canonical Entity Mapping
| Role | Surface Label | Canonical Target |
|---|---|---|
| comparison_entity_0 | EP | 04_knowledge_base/Expert Parallelism |
⑤ Human Review
請在 Properties 逐項確認：
- 原文 → Atomic Claim 是否忠實
- Atomic Claim → Semantic Frame 是否忠實
- Canonical Entity mapping 是否正確
- Epistemic mode 是否保留原文語氣
- 最後選擇
review_action
Review state
Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。