clm_6a8862de91a94e3389b400ded2677460_r1

IX2-0445

① SA Source

• Source: 開啟完整 SA 文章
• Section: Disaggregated Prefill
• Line hint: 662

Context Before

As always, the tradeoff is that of throughput versus latency. TP=8 within a group means those 8 GPUs now share a batch and must synchronize every decode step, reducing effective DP degree from 64 to 8. Per-GPU batching independence on the attention side is lost. But each DP group now processes attention 8x faster per step, since the matmul is split 8 ways across the TP group. Per-token latency drops while peak concurrency also drops, sliding the configuration along the latency-throughput Pareto frontier relative to pure DEP.

Disaggregated Prefill

Evidence

Tokens are then generated or “decoded” one at a time, loading the KV cache from HBM at each decode step

Context After

In traditional single-node inference, engines interleave prefill and decode on the same GPUs. Incoming prefill requests stall in-flight decode batches, increasing both time-to-first-token and inter-token latency. Chunked prefill mitigates this by breaking long prefills into smaller pieces, but the fundamental resource contention remains. Disaggregated prefill eliminates this entirely!

② Atomic Claim

之後 tokens 會逐一生成或 decode；每個 decode step 都需從 HBM 載入 KV cache。

• Epistemic Mode: ASSERTED
• Mapping Status: COMPLETE

③ Semantic Frame

{
  "attribute": "COUNT",
  "context_nodes": [
    {
      "id": "04_knowledge_base/HBM",
      "label": "HBM"
    },
    {
      "id": "04_knowledge_base/KV cache",
      "label": "KV cache"
    }
  ],
  "entity": {
    "id": "04_knowledge_base/Decode",
    "label": "decode"
  },
  "frame_type": "ATTRIBUTE",
  "qualifiers": {
    "condition_text": null,
    "numeric_mentions": [],
    "temporal_mentions": []
  },
  "value": {
    "numeric_mentions": [],
    "value_text": "之後 tokens 會逐一生成或 decode；每個 decode step 都需從 HBM 載入 KV cache。"
  }
}

④ Canonical Entity Mapping

Role	Surface Label	Canonical Target
entity	decode	Decode
context_0	HBM	HBM
context_1	KV cache	04_knowledge_base/KV cache

⑤ Human Review

請在 Properties 逐項確認：

• 原文 → Atomic Claim 是否忠實
• Atomic Claim → Semantic Frame 是否忠實
• Canonical Entity mapping 是否正確
• Epistemic mode 是否保留原文語氣
• 最後選擇 review_action

Review state

Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。