LLMs

Model science: how language models are built, scaled, trained, and compressed.

Troiani is the from-scratch build — a Mamba-3-dominant hybrid under 1B parameters.

• Architecture — transformers, attention, hybrids and SSMs
• Scaling — compute efficiency and power laws
• Pre-training — data, optimization, training tricks
• Mid-training — SFT, RLHF, tools and agents
• Compression — quantization, pruning, distillation
• Retrieval & RAG — embeddings and grounded generation