SA260324R-0126_r1

SA260324R-0126

① SA Source

• Source: 開啟完整 SA 文章
• Section: Speculative Decoding
• Line hint: 161

Context Before

For a decoding step of context N tokens, adding k additional tokens during forward pass (a warm prefill of k new tokens) marginally increases the latency when k << N. Using this property, speculative decoding uses a small draft model or MTP layers to predict k new tokens, saving time since small models have lower latency per decode step. To verify the draft tokens, the main model only needs one warm prefill of k new tokens, at the latency cost of roughly a single decode step.

Evidence

Speculative decoding usually boosts output token per decode step by 1.5 to 2 tokens, depending on the draft model / MTP accuracy.

Context After

With its low latency capabilities, LPUs can further increase the latency savings and improve throughput.

② Atomic Claim

Speculative decoding 通常能把每個 decode step 的 output token 提升到 1.5～2 個，實際幅度取決於 draft model／MTP 的準確率。

• Epistemic Mode: ASSERTED
• Mapping Status: PARTIAL

③ Semantic Frame

{
  "attribute": "OUTPUT_TOKENS_PER_DECODE_STEP",
  "context_nodes": [
    {
      "id": "04_knowledge_base/Decode",
      "label": "decode"
    },
    {
      "id": "04_knowledge_base/Multi-Token Prediction",
      "label": "MTP"
    }
  ],
  "entity": {
    "id": "04_knowledge_base/Speculative Decoding",
    "label": "Speculative decoding"
  },
  "frame_type": "ATTRIBUTE",
  "qualifiers": {
    "condition_text": "取決於 draft model／MTP 準確率；draft model 沒有 canonical entity",
    "numeric_mentions": [
      "1.5",
      "2"
    ],
    "temporal_mentions": []
  },
  "value": {
    "numeric_mentions": [
      "1.5",
      "2"
    ],
    "value_text": "通常可提升到每個 decode step 1.5～2 個 output token。"
  }
}

④ Canonical Entity Mapping

Role	Surface Label	Canonical Target
entity	Speculative decoding	04_knowledge_base/Speculative Decoding
context_node	decode	Decode
context_node	MTP	04_knowledge_base/Multi-Token Prediction

⑤ Human Review

請在 Properties 逐項確認：

• 原文 → Atomic Claim 是否忠實
• Atomic Claim → Semantic Frame 是否忠實
• Canonical Entity mapping 是否正確
• Epistemic mode 是否保留原文語氣
• 最後選擇 review_action

Review state

Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。