Skip to content

SJ The Scaling Journey

GitHub LinkedIn Substack

Select theme

Writing
- Overview
- Essays
  - Getting Started with The Scaling Journey
- Notes
  - Notes
LLMs
- Overview
- Troiani
- Architecture
  - Architecture Variants
  - Attention Mechanisms
- Scaling
  - Compute Efficiency
  - Power Laws in LLMs
- Pre-training
- Mid-training
- Compression
- Retrieval & RAG
  - Embeddings & Indexing
  - RAG Systems
World Models
- Overview
- Foundations
- Prediction
  - Dynamics Models
  - Visual Prediction
- Workloads
- Evaluation
- Research notes
AI Infra
Me

GitHub LinkedIn Substack

Select theme

On this page

Overview

On this page

Overview

Kernels & compilers

The layer under frameworks: GPU kernels, ML compilation, and classical compilers (LLVM, OpenMP, IR).

Notes coming.

Previous
Data plane Next
Deployment