2025-12-04_aws-trainium3-deep-dive-a-potential::TRN3-0028
① SA Source
- Source: 開啟完整 SA 文章
- Section:
2D/3D Torus vs Switched Scale Up Network - Line hint:
333
Context Before
2D/3D Torus vs Switched Scale Up Network
Before we describe the exact topology of the new switched scale-up network – let’s first explain why AWS has chosen to pivot from a 2D/3D Torus into a switched architecture. The NeuronLinkv3 scale up topology for the Trainium2 NL26 2D Torus and Trainium2 NL32x2 3D Torus Servers is a 2D mesh Torus and 3D mesh Torus respectively (as the names would suggest!). However, keeping a Torus topology is not optimized for frontier Mixture of Experts (MoE) models which require all-to-all collectives. In contrast, dense models do not make extensive use of all-to-all collectives, meaning that a switched fabric does not have much of a perf advantage when used with dense models, while having higher TCO.
Evidence
With a 3D Torus architecture, because of the oversubscription between chips within the scale-up domain, as message sizes grow from 16KB to 1MB (i.e. as you increase batch size), the scale-up network suddenly becomes bandwidth limited due to oversubscription. In contrast, with the Trainium3’s unique switched topology, oversubscription does not occur even though the first generation of Trainium3’s switched network is not a flat one-tier switched topology.
Context After
When it comes to prefill, the larger scaleup Trainium3 NL72x2 Switched scale-up topology does not provide meaningful benefit as prefill is generally compute bound, and a larger topology is mainly useful for wide expert parallelism during decode. For the decode phase when working with frontier MoE models with 2-3 trillion total parameters, the Trainium3 NL32x2 Switched’s scale-up network will be more than enough, but when working with frontier MoE models with more than 4 trillion total parameters, there will be a meaningful benefit from deploying these models on the larger Trainium3 NL72x2 Switched’s larger scale-up world size.
The Trainium3 NL32x2 Switched and Trainium3 NL72x2 Switched take the approach of ultimately providing an all-to-all scale-up switching solution but AWS’s mantra of pursuing fast time to market at a low TCO has led them to decide to build a network architecture that is compatible with different generations of scale up switches available at the given time.
② Atomic Claim
3D Torus 在 message size 16KB→1MB 時會因 oversubscription 變成 bandwidth-limited；Trainium3 switched topology 則不會發生此 oversubscription。
- Epistemic Mode:
ASSERTED - Mapping Status:
COMPLETE
③ Semantic Frame
{
"comparison_expression": "3D Torus becomes bandwidth-limited from 16KB to 1MB messages; Trainium3 switched topology avoids oversubscription",
"entities": [
{
"id": "04_knowledge_base/3D Torus",
"label": "3D Torus"
},
{
"id": "04_knowledge_base/Trainium",
"label": "Trainium3"
},
{
"id": "04_knowledge_base/Scale-up network",
"label": "Scale-up network"
}
],
"frame_type": "COMPARISON",
"metric": "oversubscription_behavior",
"operator": "SWITCHED_AVOIDS_OVERSUBSCRIPTION",
"qualifiers": {
"condition_text": null,
"numeric_mentions": [
"16KB",
"1MB"
],
"temporal_mentions": []
}
}④ Canonical Entity Mapping
| Role | Surface Label | Canonical Target |
|---|---|---|
| comparison_entity_0 | 3D Torus | 04_knowledge_base/3D Torus |
| comparison_entity_1 | Trainium3 | Trainium |
| comparison_entity_2 | Scale-up network | 04_knowledge_base/Scale-up network |
⑤ Human Review
請在 Properties 逐項確認：
- 原文 → Atomic Claim 是否忠實
- Atomic Claim → Semantic Frame 是否忠實
- Canonical Entity mapping 是否正確
- Epistemic mode 是否保留原文語氣
- 最後選擇
review_action
Review state
Markdown 內文不是正式 approval。只有 Apply bridge 寫入的 Decision Ledger event 才是正式決策。