IX2-0490
① SA Source
- Source: 開啟完整 SA 文章
- Section:
Optimizing Inference with Wide EP + Disaggregated Serving - Line hint:
696
Context Before
Source: SemiAnalysis InferenceX ↗
Evidence
Prefill is the bottleneck as each request requires a forward pass of 8192 input tokens, which is computationally expensive
Context After
On the low interactivity end of the frontier, there are fewer concurrent requests in flight, so a single prefill instance can keep pace with incoming demand. Yet each request still requires 1024 decode steps, and at high interactivity those steps must be fast. Recipes in this region shift to more decode nodes than prefill (1P3D, 1P4D), with each decode instance running TEP at low batch size. Tensor parallelism on attention minimizes per-step latency by sharding the computation across all GPUs in the instance, while expert parallelism handles MoE routing at the moderate batch sizes where EP load balance is sufficient. Multiple small-batch decode instances, rather than fewer large-batch ones, keep per-token latency low while still providing enough concurrent serving capacity.
② Atomic Claim
此時 Prefill 是瓶頸，因為每個 request 都需要對 8192 個 input tokens 做一次 forward pass，運算成本很高。
- Epistemic Mode:
ASSERTED - Mapping Status:
COMPLETE
③ Semantic Frame
{
"attribute": "COST",
"context_nodes": [],
"entity": {
"id": "04_knowledge_base/Prefill",
"label": "Prefill"
},
"frame_type": "ATTRIBUTE",
"qualifiers": {
"condition_text": null,
"numeric_mentions": [
"8192"
],
"temporal_mentions": []
},
"value": {
"numeric_mentions": [
"8192"
],
"value_text": "此時 Prefill 是瓶頸，因為每個 request 都需要對 8192 個 input tokens 做一次 forward pass，運算成本很高。"
}
}④ Canonical Entity Mapping
| Role | Surface Label | Canonical Target |
|---|---|---|
| entity | Prefill | Prefill |
⑤ Human Review
請在 Properties 逐項確認：
- 原文 → Atomic Claim 是否忠實
- Atomic Claim → Semantic Frame 是否忠實
- Canonical Entity mapping 是否正確
- Epistemic mode 是否保留原文語氣
- 最後選擇
review_action
Review state
Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。