Skip to content

SJ The Scaling Journey

GitHub LinkedIn Substack

Select theme

Writing
- Overview
- Essays
  - Getting Started with The Scaling Journey
- Notes
  - Notes
LLMs
- Overview
- Troiani
- Architecture
  - Architecture Variants
  - Attention Mechanisms
- Scaling
  - Compute Efficiency
  - Power Laws in LLMs
- Pre-training
- Mid-training
- Compression
- Retrieval & RAG
  - Embeddings & Indexing
  - RAG Systems
World Models
- Overview
- Foundations
- Prediction
  - Dynamics Models
  - Visual Prediction
- Workloads
- Evaluation
- Research notes
AI Infra
Me

GitHub LinkedIn Substack

Select theme

On this page

Overview

On this page

Overview

Cluster & platform

How the cluster is assembled: Kubernetes, Karpenter, Kueue, GPU operators, GitOps, and why research and serving sometimes share a cluster.

Notes coming.

Previous
Inference systems Next
Data plane