NIEK2-0125
① SA Source
- Source: 開啟完整 SA 文章
- Section:
Speculative Decoding - Line hint:
137
Context Before
Speculative Decoding
A different way LPUs could improve decode phase latencies is by accelerating a speculative decoding setup, where we deploy draft models or Multi-Token Prediction (MTP) layers onto LPUs.
Evidence
For a decoding step of context N tokens, adding k additional tokens during forward pass (a warm prefill of k new tokens) marginally increases the latency when k << N
Context After
Source: SemiAnalysis
② Atomic Claim
對具有 N 個 context tokens 的 decoding step 而言，若 k 遠小於 N，在 forward pass 中加入 k 個額外 token（即對 k 個新 token 做 warm prefill）只會小幅增加延遲。
- Epistemic Mode:
HYPOTHETICAL - Mapping Status:
COMPLETE
③ Semantic Frame
{
"attribute": "LATENCY",
"context_nodes": [],
"entity": {
"id": "04_knowledge_base/Prefill",
"label": "prefill"
},
"frame_type": "ATTRIBUTE",
"qualifiers": {
"condition_text": null,
"numeric_mentions": [],
"temporal_mentions": []
},
"value": {
"numeric_mentions": [],
"value_text": "對具有 N 個 context tokens 的 decoding step 而言，若 k 遠小於 N，在 forward pass 中加入 k 個額外 token（即對 k 個新 token 做 warm prefill）只會小幅增加延遲。"
}
}④ Canonical Entity Mapping
| Role | Surface Label | Canonical Target |
|---|---|---|
| entity | prefill | Prefill |
⑤ Human Review
請在 Properties 逐項確認：
- 原文 → Atomic Claim 是否忠實
- Atomic Claim → Semantic Frame 是否忠實
- Canonical Entity mapping 是否正確
- Epistemic mode 是否保留原文語氣
- 最後選擇
review_action
Review state
Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。