Power Laws in LLMs
Empirical power laws that describe model performance scaling.
Key Concepts
Section titled “Key Concepts”- Loss vs Scale relationships
- Predictive scaling
- Extrapolation capabilities
To Explore
Section titled “To Explore”- Emergent capabilities at scale
- Loss landscape changes
- Downstream task scaling