IX2-0446
① SA Source
- Source: 開啟完整 SA 文章
- Section:
Disaggregated Prefill - Line hint:
662
Context Before
As always, the tradeoff is that of throughput versus latency. TP=8 within a group means those 8 GPUs now share a batch and must synchronize every decode step, reducing effective DP degree from 64 to 8. Per-GPU batching independence on the attention side is lost. But each DP group now processes attention 8x faster per step, since the matmul is split 8 ways across the TP group. Per-token latency drops while peak concurrency also drops, sliding the configuration along the latency-throughput Pareto frontier relative to pure DEP.
Disaggregated Prefill
Evidence
This is a memory-intensive process as the growing KV cache is constantly being loaded
Context After
In traditional single-node inference, engines interleave prefill and decode on the same GPUs. Incoming prefill requests stall in-flight decode batches, increasing both time-to-first-token and inter-token latency. Chunked prefill mitigates this by breaking long prefills into smaller pieces, but the fundamental resource contention remains. Disaggregated prefill eliminates this entirely!
② Atomic Claim
- Epistemic Mode:
ASSERTED - Mapping Status:
PARTIAL
③ Semantic Frame
{
"attribute": "UNSPECIFIED_ATTRIBUTE",
"context_nodes": [
{
"id": "04_knowledge_base/Decode",
"label": "decode"
}
],
"entity": {
"id": "04_knowledge_base/KV cache",
"label": "KV cache"
},
"frame_type": "ATTRIBUTE",
"qualifiers": {
"condition_text": null,
"numeric_mentions": [],
"temporal_mentions": []
},
"value": {
"numeric_mentions": [],
"value_text": "由於持續載入不斷成長的 KV cache，decode 是 memory-intensive process。"
}
}④ Canonical Entity Mapping
| Role | Surface Label | Canonical Target |
|---|---|---|
| entity | KV cache | 04_knowledge_base/KV cache |
| context_0 | decode | Decode |
⑤ Human Review
請在 Properties 逐項確認：
- 原文 → Atomic Claim 是否忠實
- Atomic Claim → Semantic Frame 是否忠實
- Canonical Entity mapping 是否正確
- Epistemic mode 是否保留原文語氣
- 最後選擇
review_action
Review state
Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。