clm_ad5347f0a9dd46449a6c4c4ee0716fb4_r1

IX2-0470

① SA Source

• Source: 開啟完整 SA 文章
• Section: Optimizing Inference with Wide EP + Disaggregated Serving
• Line hint: 682

Context Before

Optimizing Inference with Wide EP + Disaggregated Serving

Wide EP and disaggregated prefill are separate techniques that are often used together to achieve Pareto optimal performance. In this section, we walk through real results from InferenceX to build intuition for which combinations of parallelism strategy, wide EP, and disaggregated prefill are appropriate at different interactivity levels.

Evidence

Take the example of DeepSeek R1 FP4 8k/1k on a single 8-GPU B200 node with TRT-LLM

Context After

At the highest interactivity levels (batch 1-16), pure TP outperforms any configuration involving EP. At low batch sizes, only a small fraction of experts activate per step. With EP, these activations are distributed unevenly across GPUs: at batch 4, only 32 of 256 experts fire, and any given GPU has roughly a low double digit percent chance of receiving zero routed tokens in a given layer. TP avoids this by sharding every expert across all GPUs, so all 8 GPUs participate equally in every expert computation regardless of which experts the router selects. We collected expert activation ratio versus batch size data while profiling DeepSeek R1, which confirms that at batch sizes 16 and below, expert activation per layer is very low.

② Atomic Claim

以單一 8-GPU B200 node 上、使用 TRT-LLM 執行 DeepSeek R1 FP4 8k/1k 為例。

• Epistemic Mode: ASSERTED
• Mapping Status: COMPLETE

③ Semantic Frame

{
  "frame_type": "NARY_RELATION",
  "participants": [
    {
      "node": {
        "id": "04_knowledge_base/GPU",
        "label": "GPU"
      },
      "role": "user_or_subject"
    },
    {
      "node": {
        "id": "04_knowledge_base/NVIDIA B200",
        "label": "B200"
      },
      "role": "used_entity"
    },
    {
      "node": {
        "id": "04_knowledge_base/Large language model",
        "label": "LLM"
      },
      "role": "used_entity"
    },
    {
      "node": {
        "id": "02_companies/DeepSeek",
        "label": "DeepSeek"
      },
      "role": "used_entity"
    },
    {
      "node": {
        "id": "04_knowledge_base/FP4",
        "label": "FP4"
      },
      "role": "used_entity"
    }
  ],
  "qualifiers": {
    "condition_text": null,
    "numeric_mentions": [
      "8",
      "8k",
      "1k"
    ],
    "temporal_mentions": []
  },
  "relation_type": "USES"
}

④ Canonical Entity Mapping

Role	Surface Label	Canonical Target
user_or_subject	GPU	GPU
used_entity	B200	04_knowledge_base/NVIDIA B200
used_entity	LLM	04_knowledge_base/Large language model
used_entity	DeepSeek	DeepSeek
used_entity	FP4	FP4

⑤ Human Review

請在 Properties 逐項確認：

• 原文 → Atomic Claim 是否忠實
• Atomic Claim → Semantic Frame 是否忠實
• Canonical Entity mapping 是否正確
• Epistemic mode 是否保留原文語氣
• 最後選擇 review_action

Review state

Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。