Deep Learning

Modules in this track

• What is Deep Learning? (free) - DL vs ML, when to use deep learning
• Neurons & Layers (premium) - Perceptrons, multi-layer networks
• PyTorch Fundamentals (premium) - Tensors, autograd, nn.Module, device management & torch.compile
• Activation Functions (premium) - ReLU, Sigmoid, Tanh, Softmax, SwiGLU
• Forward Propagation (premium) - Data flow through the network, matrix ops, GPU acceleration, Flash Attention
• Loss Functions for DL (premium) - Cross-entropy, MSE, BCE, focal loss, contrastive losses, and modern DPO/RLHF losses
• Backpropagation (premium) - The learning algorithm, chain rule, autograd, gradient debugging, and modern techniques
• Optimizers Deep Dive (premium) - SGD, Adam, AdamW, learning rate schedules, and modern optimizers
• Regularization (premium) - Dropout, L1/L2, early stopping, data augmentation & modern techniques
• Batch Normalization (premium) - BatchNorm mechanics, train vs eval mode, placement strategies, and normalization variants (LayerNorm, RMSNorm, GroupNorm)
• Training Deep Networks (premium) - Training loops, AdamW, LR scheduling, mixed precision, gradient clipping, checkpointing & debugging
• CNN Fundamentals (premium) - Convolutions, kernels, feature hierarchy, pooling, and modern CNN design (ConvNeXt, transfer learning)
• Pooling & CNN Architectures (premium) - Pooling strategies, strided convolutions, skip connections & architecture evolution from LeNet to ConvNeXt
• Embeddings & Representations (premium) - One-hot vs dense, nn.Embedding lookup, Word2Vec arithmetic, positional encoding, ViT patches, and dimension sizing
• RNN & Sequences (premium) - Hidden states, weight sharing, BPTT, vanishing gradients, and the modern RNN renaissance (Mamba, xLSTM, RWKV)
• LSTM & GRU (premium) - Gates, gradient highways, forget gate bias, LSTM vs GRU decision framework, and modern alternatives (xLSTM, Mamba)
• Attention Mechanism (premium) - Query-Key-Value, scaled dot-product, self/cross/causal attention, Flash Attention, GQA/MQA, the foundation of all modern AI
• Transformer Architecture (premium) - Modern attention, positional encoding, core blocks, encoder/decoder design, and key optimizations behind GPT-4, Claude, Llama, and Mistral.
• Transfer Learning & Fine-tuning (premium) - Feature reuse, freeze/unfreeze strategies, discriminative LR, domain adaptation, LoRA, adapters, and when not to transfer
• Modern Architectures (premium) - GANs, VAEs, Diffusion & Flow Matching, generative architectures behind DALL-E, Midjourney, FLUX.1, Sora, and Stable Diffusion
• GPU Fundamentals for Deep Learning (premium) - GPU architecture, CUDA cores, memory hierarchy, Tensor Cores, VRAM management, and profiling
• Multi-GPU & Distributed Training (premium) - Data parallelism, model parallelism, FSDP, DeepSpeed ZeRO, and training at scale
• Project: From Pixels to Production (premium) - Build a real image classifier from 60% to 93%+ accuracy through architecture, augmentation, and training diagnostics