IX2-0047
① SA Source
- Source: 開啟完整 SA 文章
- Section:
Interactivity vs Throughput Tradeoff - Line hint:
91
Context Before
In this section, we will give a brief primer on technical concepts that may help the reader better interpret results. Some readers may not need this and can skip directly to our analysis of results. We will take a deeper dive into some of these topics after the results analysis.
Interactivity vs Throughput Tradeoff
Evidence
The fundamental tradeoff with LLM inference is throughput versus latency. Interactivity (tok/s/user) describes how fast each user of a system receives tokens – it is the inverse of time per output token (TPOT). Throughput (tok/s) describes how many total tokens a system can crank out across all users. One can achieve higher total throughput by batching requests, but each request will be allocated less FLOPs and thus complete slower. This is analogous to the choice of riding a metro bus vs a race car. The metro bus serves many riders, but also makes frequent stops which takes time, but the cost of the metro bus can be amortized across many passengers. The race car can only carry one or two passengers, but it will make few if any additional stops meaning a faster travel time overall, but it is much more expensive to ride per passenger. The metro bus might make more sense for people heading to the park on a weekend, while the race car might be better for bringing a celebrity to their destination. There is no one size fits all solution.
Context After
Source: SemiAnalysis
② Atomic Claim
Interactivity（tok/s/user）代表系統中每位使用者收到 token 的速度，等同 time per output token（TPOT）的倒數。
- Epistemic Mode:
ASSERTED - Mapping Status:
COMPLETE
③ Semantic Frame
{
"attribute": "THROUGHPUT",
"context_nodes": [],
"entity": {
"id": "04_knowledge_base/Time Per Output Token",
"label": "time per output token"
},
"frame_type": "ATTRIBUTE",
"qualifiers": {
"condition_text": null,
"numeric_mentions": [],
"temporal_mentions": []
},
"value": {
"numeric_mentions": [],
"value_text": "Interactivity（tok/s/user）代表系統中每位使用者收到 token 的速度，等同 time per output token（TPOT）的倒數。"
}
}④ Canonical Entity Mapping
| Role | Surface Label | Canonical Target |
|---|---|---|
| entity | time per output token | 04_knowledge_base/Time Per Output Token |
⑤ Human Review
請在 Properties 逐項確認：
- 原文 → Atomic Claim 是否忠實
- Atomic Claim → Semantic Frame 是否忠實
- Canonical Entity mapping 是否正確
- Epistemic mode 是否保留原文語氣
- 最後選擇
review_action
Review state
Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。