2023-01-24_the-ai-brick-wall-a-practical-limit
Canonical Target Counts
| Target | Occurrences |
|---|
| 04_knowledge_base/Mixture of Experts | 24 |
| 02_companies/GOOG | 17 |
| 04_knowledge_base/Sparsity | 15 |
| 04_knowledge_base/GPU | 12 |
| 02_companies/NVDA | 12 |
| 04_knowledge_base/GPT-3 | 6 |
| 04_knowledge_base/Pruning | 5 |
| 04_knowledge_base/A100 | 5 |
| 02_companies/MSFT | 4 |
| 04_knowledge_base/Transformer architecture | 4 |
| 02_companies/OpenAI | 3 |
| 02_companies/ORCL | 2 |
| 04_knowledge_base/Triton | 2 |
| 04_knowledge_base/Microsoft Azure | 2 |
| 04_knowledge_base/PyTorch | 2 |
| 04_knowledge_base/CUDA | 2 |
| 04_knowledge_base/HGX | 2 |
| 04_knowledge_base/NVLink | 2 |
| 02_companies/DeepMind | 2 |
| 04_knowledge_base/Chinchilla Scaling | 2 |
| 04_knowledge_base/Early Exit | 2 |
| 04_knowledge_base/DGX | 2 |
| 04_knowledge_base/SXM | 2 |
| 04_knowledge_base/Dynamic Routing | 2 |
| 02_companies/CRWV | 1 |
| 04_knowledge_base/TPU | 1 |
| 04_knowledge_base/H100 | 1 |
| 04_knowledge_base/Quantization | 1 |
| 02_companies/Cerebras | 1 |
| 02_companies/SambaNova | 1 |
| 04_knowledge_base/1.6T | 1 |
| 02_companies/Anthropic | 1 |
| 04_knowledge_base/PCIe | 1 |
| 02_companies/Switch | 1 |
| 02_companies/Baidu | 1 |
| 04_knowledge_base/Sparse Attention | 1 |
| 02_companies/BE | 1 |
| 04_knowledge_base/AWS | 1 |
| 02_companies/COHR | 1 |
| 04_knowledge_base/MI300 | 1 |
| 02_companies/META | 1 |
| 02_companies/Amazon | 1 |
| 04_knowledge_base/TPUv4 | 1 |
| 04_knowledge_base/Software Stack | 1 |
| 04_knowledge_base/Reinforcement learning | 1 |
| 04_knowledge_base/TPUv5 | 1 |
| 04_knowledge_base/BERT | 1 |
| 04_knowledge_base/Model FLOPS Utilization | 1 |
| 04_knowledge_base/Token Dropping | 1 |
| 04_knowledge_base/Checkpointing | 1 |