NIEK2-0126
① SA Source
- Source: 開啟完整 SA 文章
- Section:
Speculative Decoding - Line hint:
137
Context Before
Speculative Decoding
A different way LPUs could improve decode phase latencies is by accelerating a speculative decoding setup, where we deploy draft models or Multi-Token Prediction (MTP) layers onto LPUs.
Evidence
Using this property, speculative decoding uses a small draft model or MTP layers to predict k new tokens, saving time since small models have lower latency per decode step
Context After
Source: SemiAnalysis
② Atomic Claim
利用這項特性，speculative decoding 可由較小的 draft model 或 MTP layers 預測 k 個新 token；由於小模型每個 decode step 延遲較低，因此能節省時間。
- Epistemic Mode:
ASSERTED - Mapping Status:
COMPLETE
③ Semantic Frame
{
"comparison_expression": "利用這項特性，speculative decoding 可由較小的 draft model 或 MTP layers 預測 k 個新 token；由於小模型每個 decode step 延遲較低，因此能節省時間。",
"entities": [
{
"id": "04_knowledge_base/Speculative Decoding",
"label": "speculative decoding"
},
{
"id": "04_knowledge_base/Multi-Token Prediction",
"label": "MTP"
},
{
"id": "04_knowledge_base/Decode",
"label": "decode"
}
],
"frame_type": "COMPARISON",
"metric": "LATENCY",
"operator": "LESS_THAN",
"qualifiers": {
"condition_text": null,
"numeric_mentions": [],
"temporal_mentions": []
}
}④ Canonical Entity Mapping
| Role | Surface Label | Canonical Target |
|---|---|---|
| comparison_entity_0 | speculative decoding | 04_knowledge_base/Speculative Decoding |
| comparison_entity_1 | MTP | 04_knowledge_base/Multi-Token Prediction |
| comparison_entity_2 | decode | Decode |
⑤ Human Review
請在 Properties 逐項確認：
- 原文 → Atomic Claim 是否忠實
- Atomic Claim → Semantic Frame 是否忠實
- Canonical Entity mapping 是否正確
- Epistemic mode 是否保留原文語氣
- 最後選擇
review_action
Review state
Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。