IX2-0177
① SA Source
- Source: 開啟完整 SA 文章
- Section:
Unpacking Inference Providers’ Unit Economics - Line hint:
314
Context Before
Source: OpenRouter ↗
We can then use real InferenceX data to interpolate the cost per million input/output tokens at an interactivity level of 35 tok/sec/user, which is a reasonable interactivity level given the data above.
Evidence
As we mention later in the article, this is best understood as _baseline _data and not completely representative of real-world inference, mainly because InferenceX benchmarks on random data and disables prefix caching. In other words, performance/cost will be _at least _this good. It is also important to note that there are not data points for each GPU at _each _interactivity level. Thus we cannot make _exact _comparisons at each degree of interactivity. We nevertheless think the bar chart comparisons presented below are (very) reasonable interpolations in lieu of using exact data points.
Context After
Comparing disagg+wideEP configs at this interactivity level, we see just how effective distributed inference techniques are when it comes to both perf/TCO and overall throughput. We also see how large scale up domains (like GB300 and GB200 NVL72) absolutely dominate in total throughput per GPU.
It is interesting to note that at this interactivity level (on an 8k1k workload type), the B200 can achieve the best perf/TCO when MTP is enabled. Below we also list the Total Cost of Ownership (TCO) (Owning – Hyperscaler) for each GPU:
② Atomic Claim
如文章後段說明，這些結果更適合視為 baseline，而不完全代表 real-world inference，主要因為 InferenceX 使用隨機資料做 benchmark，且關閉 prefix caching。
- Epistemic Mode:
ASSERTED - Mapping Status:
COMPLETE
③ Semantic Frame
{
"additional_nodes": [],
"frame_type": "RELATION",
"object": {
"id": "04_knowledge_base/Prefix Caching",
"label": "prefix caching"
},
"predicate": "USES",
"qualifiers": {
"condition_text": null,
"numeric_mentions": [],
"temporal_mentions": []
},
"subject": {
"id": "04_knowledge_base/InferenceX",
"label": "InferenceX"
}
}④ Canonical Entity Mapping
| Role | Surface Label | Canonical Target |
|---|---|---|
| subject | InferenceX | InferenceX |
| object | prefix caching | 04_knowledge_base/Prefix Caching |
⑤ Human Review
請在 Properties 逐項確認：
- 原文 → Atomic Claim 是否忠實
- Atomic Claim → Semantic Frame 是否忠實
- Canonical Entity mapping 是否正確
- Epistemic mode 是否保留原文語氣
- 最後選擇
review_action
Review state
Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。