Advanced

9-11 weeks•46 lessons•9 modules

1

Advanced Transformer Concepts

5 lessons

1Multi-Head Attention Deep Dive

2Layer Normalization and Residual Connections

3Tokenization (BPE, WordPiece, SentencePiece, Tiktoken)

4Model Scaling Laws

5Paper: Scaling Laws for Neural Language Models

2

Modern Architectures

5 lessons

1Llama Architecture Analysis

2Mistral and Mixture of Experts (MoE)

3Paper: LLaMA — Open and Efficient Foundation Models

4FlashAttention and Optimization Techniques

5Rotary Position Embeddings (RoPE)

3

Fine-Tuning Techniques

5 lessons

1Transfer Learning for LLMs

2Full Fine-Tuning vs Parameter-Efficient Methods

3LoRA (Low-Rank Adaptation)

4QLoRA: Quantized LoRA

5Paper: LoRA — Low-Rank Adaptation

4

Advanced Fine-Tuning

5 lessons

1Instruction Tuning

2RLHF (Reinforcement Learning from Human Feedback)

3DPO (Direct Preference Optimization)

4Paper: Training Language Models to Follow Instructions

5Project: Fine-Tune Your Own Model

5

AI Agents & Autonomous Systems

5 lessons

1ReAct: Reasoning and Acting

2Paper: ReAct — Synergizing Reasoning and Acting

3Tool Use, Function Calling, and Computer Use

4Multi-Agent Systems

5Project: Autonomous Research Agent

6

Advanced RAG & Context

5 lessons

1Advanced Chunking Strategies

2Re-ranking and Hybrid Search

3Query Expansion Techniques

4Context Compression and Long-Context RAG

5Project: Production-Ready RAG System

7

Optimization & Deployment

5 lessons

1Quantization (GPTQ, AWQ, GGUF, FP8)

2Model Serving: vLLM, TGI, SGLang

3Prompt Caching and KV-Cache Strategies

4Cost Optimization

5Monitoring and Observability

8

Cutting-Edge Topics

6 lessons

1Mixture of Experts (MoE) in Depth

2Long-Context Models (1M+ tokens)

3Multimodal Models (Vision, Audio, Video)

4Computer-Use Agents (Claude, OpenAI Operator)

5Constitutional AI and Safety

6The Future of LLMs

9

Production Agentic Systems

5 lessons

1AutoGen: Microsoft's Multi-Agent Framework

2Agent Orchestration Patterns

3Building MCP Servers for Production

4Evaluating & Testing AI Agents

5Project: Production Multi-Agent Pipeline