IX2-0127
① SA Source
- Source: 開啟完整 SA 文章
- Section:
Disaggregated Inference Frameworks - Line hint:
200
Context Before
Source: SemiAnalysis InferenceX ↗
Disaggregated Inference Frameworks
Evidence
NVIDIA uses Dynamo for its disaggregated inference setup. Dynamo ↗ is an inference framework designed for multi-node distributed inference, featuring techniques such as prefill-decode disaggregation, request routing, and KV cache offloading. It is inference-engine agnostic, allowing us to use SGLang and TRT LLM as backends in our benchmark. For AMD, we use SGLang with two different KV cache transfer frameworks: MoRI and Mooncake. MoRI ↗ is a high-performance communication interface focusing on RDMA and GPU integration, offering applications such as network collective operations and expert parallel kernels. Mooncake, which recently joined the PyTorch ecosystem ↗, supports prefill-decode disaggregation and many fault tolerant multi-node features.
Context After
DeepSeek Disagg +WideEP Results Deep Dive
At almost all interactivity levels, disagg outperform aggregated inference (grey lines) in terms of total token throughput per GPU. Multi-node disaggregrated prefill framemogs single node aggregrated serving.
② Atomic Claim
Dynamo 是為 multi-node distributed inference 設計的 inference framework，支援 prefill-decode disaggregation、request routing 與 KV cache offloading 等技術。
- Epistemic Mode:
ASSERTED - Mapping Status:
COMPLETE
③ Semantic Frame
{
"frame_type": "NARY_RELATION",
"participants": [
{
"node": {
"id": "04_knowledge_base/Prefill",
"label": "prefill"
},
"role": "supporting_subject"
},
{
"node": {
"id": "04_knowledge_base/Decode",
"label": "decode"
},
"role": "supported_entity"
},
{
"node": {
"id": "04_knowledge_base/KV Cache Offloading",
"label": "KV cache offloading"
},
"role": "supported_entity"
}
],
"qualifiers": {
"condition_text": null,
"numeric_mentions": [],
"temporal_mentions": []
},
"relation_type": "SUPPORTS"
}④ Canonical Entity Mapping
| Role | Surface Label | Canonical Target |
|---|---|---|
| supporting_subject | prefill | Prefill |
| supported_entity | decode | Decode |
| supported_entity | KV cache offloading | 04_knowledge_base/KV Cache Offloading |
⑤ Human Review
請在 Properties 逐項確認：
- 原文 → Atomic Claim 是否忠實
- Atomic Claim → Semantic Frame 是否忠實
- Canonical Entity mapping 是否正確
- Epistemic mode 是否保留原文語氣
- 最後選擇
review_action
Review state
Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。