Click any module to expand. Every module is implemented in PyTorch with GPU training on Google Colab (free). The module starts with neural network fundamentals and progresses to the 2025–26 frontier: Vision Transformers, LLM fine-tuning with LoRA, multimodal models, and RAG architecture.
Tensors, autograd, custom datasets, and building neural networks from scratch
Tensors: creation (torch.tensor, zeros, ones, randn), operations, device management (CPU ↔ GPU with .to('cuda')). Autograd: automatic differentiation — requires_grad=True, .backward(), .grad. Computational graph: how PyTorch tracks operations for backpropagation. The foundation of every PyTorch model: tensors + autograd + GPU = fast training.
torch.utils.data.Dataset: custom dataset class (__len__, __getitem__). DataLoader: batching, shuffling, num_workers for parallel loading. torchvision.transforms: Compose, Resize, ToTensor, Normalize, RandomHorizontalFlip. Data augmentation pipeline for training vs validation (augment train only). The data pipeline every PyTorch project needs — custom datasets for ANY data format (images, text, tabular).
nn.Module: the base class for all models. __init__ (define layers), forward (define computation). Common layers: nn.Linear, nn.ReLU, nn.Dropout, nn.BatchNorm1d. nn.Sequential for simple architectures. Model parameters: model.parameters(), count with sum(p.numel()). Saving/loading: torch.save(model.state_dict()), model.load_state_dict(). Building an MLP classifier from scratch on tabular data.
The training loop: optimizer.zero_grad() → output = model(x) → loss = criterion(output, y) → loss.backward() → optimizer.step(). Validation loop: with torch.no_grad(). Loss functions: CrossEntropyLoss (classification), MSELoss (regression), BCEWithLogitsLoss (binary). Optimizers: SGD (+ momentum), Adam, AdamW. Learning rate scheduling: StepLR, CosineAnnealingLR, OneCycleLR. Writing the training loop from scratch teaches what frameworks abstract away.
Google Colab: free GPU (T4) access. Runtime → Change runtime type → GPU. Moving tensors and model to GPU: .to('cuda'). Mixed precision training: torch.cuda.amp for 2x speed with half-precision. Gradient accumulation for training with larger effective batch sizes on limited GPU memory. torch.cuda.memory_summary() for debugging OOM. The practical GPU training skills every DL practitioner needs.
W&B (wandb): log metrics, hyperparameters, model architecture, sample predictions. wandb.init(), wandb.log(). Visualise training curves (loss, accuracy) across runs. Hyperparameter sweeps with wandb.sweep. Model checkpointing. Compare experiments. Why W&B is the DL experiment tracking standard — used at OpenAI, Google DeepMind, and most AI research labs. Free tier is sufficient for learning.
The techniques that make deep models generalise — dropout, batch norm, scheduling, and augmentation
ReLU (default hidden), LeakyReLU, GELU (used in Transformers), Sigmoid (binary output), Softmax (multiclass), Swish/SiLU. Vanishing/exploding gradients: why deep networks are hard to train. Weight initialisation: Xavier/Glorot (sigmoid/tanh), He/Kaiming (ReLU). Why initialisation matters — wrong init → dead neurons or divergence.
Dropout: randomly zero neurons during training (p=0.5 typical). Batch normalisation: normalise layer inputs → faster training + mild regularisation. Layer normalisation: used in Transformers (normalise across features, not batch). L2 regularisation via weight_decay in optimizer. Label smoothing: soften one-hot targets. Data augmentation as regularisation. The toolkit for preventing overfitting in deep models.
Warm-up: start with tiny LR, ramp up (critical for Transformers). Cosine annealing: smooth LR decay. OneCycleLR: super-convergence (train faster with high max LR). ReduceLROnPlateau: reduce when validation loss stops improving. Early stopping: patience-based training termination. "Finding the right learning rate" with LR Finder. The scheduling strategy that can double training efficiency.
Mixed precision (FP16/BF16): 2x faster training, half memory usage. torch.cuda.amp: autocast + GradScaler. BFloat16 on newer GPUs (A100, H100). Gradient checkpointing: trade compute for memory. Distributed training: DataParallel (single machine, multi-GPU), DistributedDataParallel (multi-machine). DeepSpeed for large model training. The techniques that enable training models that don't fit in single-GPU memory.
Image classification, object detection, segmentation — from ConvNets to ViT
Convolution: learnable filters extract spatial features (edges → textures → shapes → objects). Kernel/filter, stride, padding (same/valid). Feature maps. Pooling: MaxPool (take max), AvgPool (take average), GlobalAvgPool (entire feature map → single value). Architecture: Conv-ReLU-Pool stacks → Flatten → Dense. Receptive field: how deep layers "see" larger image regions. Building a CNN from scratch on CIFAR-10.
Architecture evolution: LeNet (1998, simple) → AlexNet (2012, deep + ReLU + dropout) → VGG (2014, 3×3 everywhere) → GoogLeNet/Inception (2014, parallel filters) → ResNet (2015, skip connections — solved vanishing gradients for 100+ layers) → EfficientNet (2019, compound scaling). Why ResNet's skip connections were the breakthrough. How to read architecture papers.
Pretrained models: trained on ImageNet (14M images, 1000 classes). Transfer: use pretrained features, replace final classifier. Fine-tuning strategies: freeze all → unfreeze top layers → unfreeze all with small LR. torchvision.models: resnet50(pretrained=True), efficientnet_v2_s. timm library for 700+ pretrained models. Practical: classify custom dataset (medical images, food, products) with 50 training images using transfer learning.
ViT: split image into patches → flatten → linear projection → positional encoding → Transformer encoder → classification head. Why ViT works: self-attention captures global relationships (CNNs are local). ViT vs CNN: ViT needs more data (or pretrained), but scales better with compute. DeiT (data-efficient ViT), Swin Transformer (hierarchical windows). The architecture replacing CNNs for many vision tasks in 2025.
Object detection: YOLO (single-shot, real-time), Faster R-CNN (two-stage, accurate). YOLO evolution: YOLOv5 → YOLOv8 → YOLOv11 (Ultralytics). Segmentation: semantic (per-pixel class), instance (separate objects), panoptic (both). U-Net (medical), Mask R-CNN, SAM (Segment Anything Model — Meta, 2023). Overview depth — practical with YOLO and SAM, conceptual for others.
Grad-CAM: gradient-weighted class activation maps — visualise WHERE the model is looking. "The model classified this as a cat because it focused on the face region." Crucial for debugging: model using wrong features (background instead of object). Grad-CAM for CNNs, attention visualisation for ViTs. Model interpretability is required for medical, legal, and safety-critical applications.
Processing sequential data — and how attention replaced recurrence
RNN: hidden state passed through time steps. Vanishing gradient: gradients shrink exponentially over long sequences → can't learn long dependencies. LSTM: forget gate (what to discard), input gate (what to add), output gate (what to expose). GRU: simplified LSTM (2 gates instead of 3). Bidirectional: process forward AND backward. Building a text classifier with LSTM from scratch in PyTorch.
Encoder-decoder: compress input sequence → generate output sequence. Applications: translation, summarisation, question answering. Attention mechanism: instead of one fixed context vector, attend to ALL encoder hidden states weighted by relevance. Bahdanau (additive) vs Luong (multiplicative) attention. Self-attention: attend within the same sequence. Attention IS the mechanism that led to Transformers — understanding it is essential.
Time series features: lag features, rolling statistics, date features. LSTM/GRU for time series: sequence of past values → predict next value(s). Temporal Convolutional Networks (TCN): 1D convolutions over time. Temporal Fusion Transformer (TFT): attention-based time series (state-of-the-art). Practical: stock price, energy demand, weather forecasting. When DL beats classical (long sequences, complex patterns) and when it doesn't (short series, few features).
The architecture that powers GPT, BERT, and every modern AI model
Self-attention: Query, Key, Value matrices → attention scores → weighted sum. Multi-head attention: multiple attention heads capture different relationships. Positional encoding: sine/cosine or learnable. Feed-forward network after attention. Layer normalisation (pre-norm vs post-norm). Encoder (bidirectional — BERT) vs decoder (autoregressive — GPT). The architecture that changed AI — implemented from scratch in PyTorch.
Pre-training: Masked Language Model (predict masked tokens) + Next Sentence Prediction. Fine-tuning: add classification head for downstream tasks. BERT variants: DistilBERT (smaller, 97% performance), RoBERTa (improved training), ALBERT (parameter sharing). Tokenisation: WordPiece. Fine-tuning BERT for sentiment analysis with Hugging Face Trainer API. The encoder model that dominates text classification, NER, and question answering.
GPT architecture: decoder-only Transformer with causal (masked) self-attention. Pre-training: next-token prediction on massive text. Scaling laws: more data + more parameters = better performance. GPT-2 → GPT-3 → GPT-4 evolution. Emergent abilities at scale: in-context learning, chain-of-thought reasoning. Tokenisation: BPE (Byte-Pair Encoding). The architecture behind ChatGPT, Claude, and every modern LLM.
pipeline() for zero-shot classification, NER, summarisation, translation — 3 lines of code. AutoModel, AutoTokenizer: load any model from Hub. Trainer API: fine-tune with minimal code. Datasets library: load standard benchmarks. Model Hub: browse, download, share models. The library that democratised NLP — 500K+ models available. Practical: fine-tune a text classifier, deploy via API.
LoRA, QLoRA, PEFT, RLHF — the techniques for adapting foundation models
Full fine-tuning: update ALL model parameters (expensive, needs lots of GPU memory). PEFT: update only a SMALL subset of parameters. LoRA (Low-Rank Adaptation): add trainable low-rank matrices to attention layers — train 0.1–1% of parameters, match full fine-tuning quality. QLoRA: 4-bit quantised base model + LoRA adapters — fine-tune 7B parameter models on a single consumer GPU. The technique that makes LLM customisation accessible.
Base model → instruction-tuned model: teach the LLM to follow instructions. Dataset format: {"instruction": "Summarise this...", "input": "...", "output": "..."}. SFT (Supervised Fine-Tuning) with LoRA on Hugging Face. Alpaca, Dolly, OpenAssistant datasets. Creating custom instruction datasets for domain-specific tasks. The process behind every "chat model" — from base GPT to ChatGPT.
Reinforcement Learning from Human Feedback: SFT → Reward Model training → PPO optimisation. Why RLHF: base models can generate harmful, biased, or unhelpful content — RLHF aligns with human preferences. DPO (Direct Preference Optimisation): simpler alternative to PPO (no separate reward model). Constitutional AI. Overview depth — understanding the concepts and why alignment matters, not implementing PPO from scratch.
Perplexity: how "surprised" the model is by the test data. BLEU, ROUGE for text generation quality. Human evaluation: preference ranking, Chatbot Arena. Benchmarks: MMLU (knowledge), HellaSwag (commonsense), HumanEval (coding). LLM evaluation is hard — no single metric captures "good." Understanding evaluation helps choose the right model for the right task.
Creating new data — images, text, and synthetic data generation
Autoencoder: encoder compresses → latent space → decoder reconstructs. Use cases: dimensionality reduction, denoising, anomaly detection (high reconstruction error = anomaly). Variational Autoencoder (VAE): learn a latent distribution, sample from it → generate new data. Reparameterisation trick. VAE for generating faces, molecules, data augmentation.
GAN: generator creates fake data, discriminator distinguishes real from fake — adversarial training. Training challenges: mode collapse, training instability. GAN variants: DCGAN (convolutional), StyleGAN (high-quality face generation), Pix2Pix (image translation), CycleGAN (unpaired image translation). Conditional GANs. Practical: generate handwritten digits, face generation with StyleGAN.
Forward process: gradually add noise to data until pure noise. Reverse process: learn to denoise step by step → generate data from noise. DDPM (Denoising Diffusion Probabilistic Models). Why diffusion models surpassed GANs: more stable training, better sample diversity. Stable Diffusion architecture: U-Net + CLIP text encoder + VAE decoder. The technology behind DALL-E, Midjourney, and Stable Diffusion. Overview depth with practical experimentation.
Beyond single-modality — models that understand images, text, and audio together
CLIP (Contrastive Language-Image Pre-training): jointly train image encoder + text encoder to match images with descriptions. Zero-shot image classification: describe categories in text → CLIP matches. LLaVA (Large Language and Vision Assistant): LLM that "sees" images — visual question answering, image description. The multimodal trend: models that process MULTIPLE data types simultaneously. The frontier of AI in 2025–26.
Whisper (OpenAI): speech-to-text in 99 languages. Architecture: encoder-decoder Transformer trained on 680K hours of audio. Using Whisper for transcription: whisper.load_model(), model.transcribe(). Fine-tuning Whisper on domain-specific audio (medical, legal). Audio classification with Hugging Face. The model that made speech recognition accessible to everyone.
GNNs: neural networks on graph-structured data (social networks, molecules, citation networks). Message passing: nodes aggregate information from neighbours. GCN (Graph Convolutional Network), GAT (Graph Attention Network). PyTorch Geometric library. Use cases: drug discovery (molecular property prediction), social network analysis, recommendation systems. Overview — understanding where GNNs apply without deep implementation.
Self-supervised: learn representations WITHOUT labels. Contrastive learning: SimCLR, DINO — learn by comparing positive pairs (augmented views of same image) vs negative pairs. Masked image modelling (MAE): mask patches, predict missing patches. Why self-supervised matters: labelling data is expensive — self-supervised learns from unlimited unlabelled data. The pre-training paradigm behind foundation models.
Retrieval-Augmented Generation, embeddings, vector databases, and building AI products
Embeddings: dense vector representations of text/images. Sentence Transformers: encode text into embeddings for semantic similarity. OpenAI text-embedding-3-small. Cosine similarity for comparing embeddings. Vector databases: ChromaDB (simple), Pinecone (managed), Weaviate, FAISS (Facebook's fast similarity search). Indexing strategies: HNSW, IVF. The infrastructure that enables RAG and semantic search.
RAG architecture: embed documents → chunk → store in vector DB → user query → retrieve relevant chunks → LLM generates answer from context. Why RAG: LLMs hallucinate, RAG grounds them in YOUR data. LangChain: document loaders, text splitters, retrievers, chains. Chunking strategies: fixed-size, recursive, semantic. Evaluation: retrieval quality (recall@K), generation quality (faithfulness, relevance). Building a "chat with your documents" application.
AI agents: LLM + tools (search, code execution, API calls) + reasoning (ReAct pattern). LangChain agents, LangGraph for complex workflows. Tool use: LLM decides which tool to call based on user request. Function calling with OpenAI API. Memory: conversation history, summarisation. Limitations: hallucination in tool selection, safety concerns. The architecture powering AI assistants, coding agents, and autonomous systems.
Model compression, serving, edge deployment, and building a portfolio
Quantisation: FP32 → INT8 (4x smaller, 2–4x faster inference). Post-training quantisation (easy) vs quantisation-aware training (better). ONNX: export PyTorch model to framework-agnostic format. ONNX Runtime for optimised inference. Pruning: remove unimportant weights (structured vs unstructured). Knowledge distillation: train smaller "student" from larger "teacher." The techniques that make DL models deployable on real hardware.
TorchServe: production PyTorch model serving (batching, multi-model, metrics). FastAPI: custom REST API for model inference. Gradio: interactive web demo in 5 lines (upload image → get prediction). Hugging Face Spaces: free model hosting with Gradio/Streamlit. Batch prediction vs real-time inference trade-offs. The deployment stack: develop (Colab) → serve (FastAPI/TorchServe) → demo (Gradio) → host (HF Spaces).
Edge: run models on mobile, embedded, browser — not cloud. TensorFlow Lite (mobile), ONNX Runtime Mobile, Core ML (iOS), TensorRT (NVIDIA edge). Model size constraints: ≤50MB for mobile. Why edge matters: latency (no network round-trip), privacy (data stays on device), cost (no cloud compute). Use cases: camera filters, voice assistants, autonomous vehicles. Overview — understanding the edge landscape.
Portfolio structure: 4–5 projects showing progression (image classification → NLP → generative → RAG → deployment). Each project: Kaggle notebook + GitHub repo + deployed demo (Gradio/HF Spaces). Hugging Face model cards: document model, training data, limitations. W&B reports: visualise experiments. GitHub README with architecture diagrams, results, and live demo links. Papers With Code for finding and reproducing research. The portfolio that gets DL/AI Engineer interviews.
Fine-tune Vision Transformer on custom dataset using LoRA (0.5% parameters), Grad-CAM visualisation, W&B logging, deployed on HF Spaces.
Sentiment analysis / topic classification: fine-tune BERT with Hugging Face Trainer, evaluate on test set, deploy via FastAPI.
Embed documents → ChromaDB → query → LLM generates answers with citations. LangChain pipeline with Streamlit frontend.
Fine-tune Stable Diffusion on custom style/domain using LoRA, generate images from text prompts, deploy on HF Spaces.
Trainers implement models live on Google Colab with GPU — from tensor operations to training loops to W&B logging. Students train alongside on their own GPU runtime.
Each week: implement a model, train it, evaluate it, log experiments in W&B. Progressive difficulty: MLP → CNN → BERT fine-tune → LoRA → RAG.
Trainers review: model architecture choices, training configurations, experiment logs, and results interpretation. The feedback that builds DL engineering intuition.
Students deploy models via Gradio on Hugging Face Spaces. Interactive demos with live URLs. The portfolio that makes interviewers say "let me try your model."
AI Engineer, DL Engineer, ML Engineer — roles at Google, Amazon, NVIDIA, startups, and every company building AI products. Deep learning is the technical foundation of every AI product: recommendation systems, search, vision, language, generation. DL skills command the highest technical salaries in the industry.
Every company wants to customise LLMs for their domain — but full fine-tuning is expensive. LoRA/QLoRA makes it accessible: fine-tune 7B parameter models on a single GPU. "How would you adapt this LLM for our use case?" → LoRA fine-tuning. This single skill is the most in-demand DL capability in the current job market.
Every enterprise AI application uses RAG: "chat with your documents," "internal knowledge base Q&A," "customer support with company-specific answers." RAG is the most practical GenAI architecture — and the most hired-for. Students who can build RAG pipelines with LangChain and vector databases are immediately hireable.
A Gradio demo on Hugging Face Spaces where the interviewer can upload an image and see your model's prediction — this is the strongest possible DL portfolio piece. "Try my model" beats "look at my notebook" every time. 4–5 deployed projects with W&B experiment logs demonstrate production DL engineering skills.
E-305, Industrial Area, Sahibzada Ajit Singh Nagar, Punjab 160055, IN
