IX2-0178
① SA Source
- Source: 開啟完整 SA 文章
- Section:
Unpacking Inference Providers’ Unit Economics - Line hint:
314
Context Before
Source: OpenRouter ↗
We can then use real InferenceX data to interpolate the cost per million input/output tokens at an interactivity level of 35 tok/sec/user, which is a reasonable interactivity level given the data above.
Evidence
As we mention later in the article, this is best understood as _baseline _data and not completely representative of real-world inference, mainly because InferenceX benchmarks on random data and disables prefix caching. In other words, performance/cost will be _at least _this good. It is also important to note that there are not data points for each GPU at _each _interactivity level. Thus we cannot make _exact _comparisons at each degree of interactivity. We nevertheless think the bar chart comparisons presented below are (very) reasonable interpolations in lieu of using exact data points.
Context After
Comparing disagg+wideEP configs at this interactivity level, we see just how effective distributed inference techniques are when it comes to both perf/TCO and overall throughput. We also see how large scale up domains (like GB300 and GB200 NVL72) absolutely dominate in total throughput per GPU.
It is interesting to note that at this interactivity level (on an 8k1k workload type), the B200 can achieve the best perf/TCO when MTP is enabled. Below we also list the Total Cost of Ownership (TCO) (Owning – Hyperscaler) for each GPU:
② Atomic Claim
關於 InferenceX：換句話說，實際 production 的 performance/cost 至少應該不會比這個 baseline 更差。
- Epistemic Mode:
ASSERTED - Mapping Status:
COMPLETE
③ Semantic Frame
{
"attribute": "COST",
"context_nodes": [],
"entity": {
"id": "04_knowledge_base/InferenceX",
"label": "InferenceX"
},
"frame_type": "ATTRIBUTE",
"qualifiers": {
"condition_text": null,
"numeric_mentions": [],
"temporal_mentions": []
},
"value": {
"numeric_mentions": [],
"value_text": "關於 InferenceX：換句話說，實際 production 的 performance/cost 至少應該不會比這個 baseline 更差。"
}
}④ Canonical Entity Mapping
| Role | Surface Label | Canonical Target |
|---|---|---|
| entity | InferenceX | InferenceX |
⑤ Human Review
請在 Properties 逐項確認：
- 原文 → Atomic Claim 是否忠實
- Atomic Claim → Semantic Frame 是否忠實
- Canonical Entity mapping 是否正確
- Epistemic mode 是否保留原文語氣
- 最後選擇
review_action
Review state
Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。