Compute Efficiency
Core principles of scaling laws and compute efficiency in training.
Key Concepts
Section titled “Key Concepts”- Chinchilla Scaling
- Compute-optimal allocation
- Model vs Data scaling trade-offs
To Explore
Section titled “To Explore”- Loss scaling behavior
- FLOPs vs parameters
- Energy efficiency considerations