SA260324R-0125_r1

SA260324R-0125

① SA Source

• Source: 開啟完整 SA 文章
• Section: Speculative Decoding
• Line hint: 159

Context Before

For a decoding step of context N tokens, adding k additional tokens during forward pass (a warm prefill of k new tokens) marginally increases the latency when k << N. Using this property, speculative decoding uses a small draft model or MTP layers to predict k new tokens, saving time since small models have lower latency per decode step.

Evidence

To verify the draft tokens, the main model only needs one warm prefill of k new tokens, at the latency cost of roughly a single decode step.

Context After

Speculative decoding usually boosts output token per decode step by 1.5 to 2 tokens, depending on the draft model / MTP accuracy. With its low latency capabilities, LPUs can further increase the latency savings and improve throughput.

② Atomic Claim

為了驗證 draft token，main model 只需要對 k 個新 token 做一次 warm prefill，其 latency cost 約等於一次 decode step。

• Epistemic Mode: ASSERTED
• Mapping Status: PARTIAL

③ Semantic Frame

{
  "comparison_expression": "一次 warm prefill（k 個新 token）的 latency cost 約等於一次 decode step。",
  "entities": [
    {
      "id": "04_knowledge_base/Prefill",
      "label": "prefill"
    },
    {
      "id": "04_knowledge_base/Decode",
      "label": "decode"
    }
  ],
  "frame_type": "COMPARISON",
  "metric": "LATENCY_COST",
  "operator": "APPROX_EQUAL",
  "qualifiers": {
    "condition_text": "用於驗證 draft token；draft token 與 main model 沒有 canonical entity",
    "numeric_mentions": [
      "1",
      "k"
    ],
    "temporal_mentions": []
  }
}

④ Canonical Entity Mapping

Role	Surface Label	Canonical Target
entity	prefill	Prefill
entity	decode	Decode

⑤ Human Review

請在 Properties 逐項確認：

• 原文 → Atomic Claim 是否忠實
• Atomic Claim → Semantic Frame 是否忠實
• Canonical Entity mapping 是否正確
• Epistemic mode 是否保留原文語氣
• 最後選擇 review_action

Review state

Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。