Advanced

14-16 weeks•67 lessons•12 modules

1

Advanced Transformer Concepts

5 lessons

1Multi-Head Attention Deep Dive

2Layer Normalization and Residual Connections

3Tokenization (BPE, WordPiece, SentencePiece, Tiktoken)

4Model Scaling Laws

5Paper: Scaling Laws for Neural Language Models

2

Modern Architectures

5 lessons

1Llama Architecture Analysis

2Mistral and Mixture of Experts (MoE)

3Paper: LLaMA — Open and Efficient Foundation Models

4FlashAttention and Optimization Techniques

5Rotary Position Embeddings (RoPE)

3

Foundation Model Training

5 lessons

1How Foundation Models Are Trained

2Training Data Mixtures and Quality

3Distributed Training Systems

4Post-Training and Alignment Pipeline

5Model Release, Evaluation, and Governance

4

Fine-Tuning Techniques

6 lessons

1Transfer Learning for LLMs

2Full Fine-Tuning vs Parameter-Efficient Methods

3LoRA (Low-Rank Adaptation)

4QLoRA: Quantized LoRA

5Paper: LoRA — Low-Rank Adaptation

6Fine-Tuning Methods: SFT, PEFT, RLHF, and DPO

5

Advanced Fine-Tuning

6 lessons

1Instruction Tuning

2RLHF (Reinforcement Learning from Human Feedback)

3DPO (Direct Preference Optimization)

4Paper: Training Language Models to Follow Instructions

5Project: Fine-Tune Your Own Model

6Distillation and Synthetic Data

6

AI Agents & Autonomous Systems

9 lessons

1ReAct: Reasoning and Acting

2Paper: ReAct — Synergizing Reasoning and Acting

3Tool Use, Function Calling, and Computer Use

4Multi-Agent Systems

5Project: Autonomous Research Agent

6A2A and Multi-Agent Protocols

7Agent Memory and Planning

8Reliable Agent Control Loops

9Human-in-the-Loop Agents

7

Advanced RAG & Context

7 lessons

1Advanced Chunking Strategies

2Re-ranking and Hybrid Search

3Query Expansion Techniques

4Context Compression and Long-Context RAG

5Project: Production-Ready RAG System

6GraphRAG and Structured Knowledge Retrieval

7RAG Evaluation for Production

8

Optimization & Deployment

8 lessons

1Quantization (GPTQ, AWQ, GGUF, FP8)

2Model Serving: vLLM, TGI, SGLang

3Prompt Caching and KV-Cache Strategies

4Cost Optimization

5Monitoring and Observability

6LLM Gateways, Routing, and Fallbacks

7Speculative Decoding and Inference-Time Scaling

8Edge and On-Device LLM Inference

9

Cutting-Edge Topics

7 lessons

1Mixture of Experts (MoE) in Depth

2Long-Context Models (128K to 1M+ Tokens)

3Multimodal Models (Vision, Audio, Video)

4Computer-Use Agents (Browser and Desktop Automation)

5The Future of LLMs

6Reasoning Effort, Budgets, and Test-Time Compute

7Realtime Voice and Audio Agents

10

Production Agentic Systems

5 lessons

1AutoGen: Microsoft's Multi-Agent Framework

2Agent Orchestration Patterns

3Building MCP Servers for Production

4Evaluating & Testing AI Agents

5Project: Production Multi-Agent Pipeline

11

Security, Safety & Risk

3 lessons

1Prompt Injection Defense

2OWASP LLM Top 10 for AI Applications

3Constitutional AI and Safety

12

Research Paper Roadmap

1 lessons

1Research Paper Roadmap: From NLP Foundations to Modern LLMs