Click any module to expand. This module goes DEEPER than the Data Science module's ML coverage — algorithm internals, mathematical derivations, advanced tuning, deep learning, NLP, and production engineering. The Data Science module teaches "use XGBoost." This module teaches "understand WHY XGBoost works and HOW to make it work better."
The math BEHIND ML — taught through the lens of specific algorithms, not as abstract theory
Vectors (feature vectors, dot product = similarity measure). Matrices (data matrix, weight matrix, transformation). Matrix multiplication: why neural networks are matrix operations. Eigenvalues/eigenvectors: the math behind PCA. SVD: the math behind recommendation systems and dimensionality reduction. Not abstract math — each concept tied to a specific ML algorithm.
Derivatives: how loss changes with parameter change. Partial derivatives: gradient = vector of partial derivatives. Chain rule: the math behind backpropagation. Gradient descent: move parameters in the direction that reduces loss. Variants: batch, stochastic (SGD), mini-batch. Learning rate: too high → diverge, too low → slow. Adam optimiser: adaptive learning rates. The optimisation loop every ML model uses.
Bayes' theorem: the foundation of Naive Bayes and Bayesian methods. Conditional probability: P(spam | "free money"). Likelihood vs probability. MLE (Maximum Likelihood Estimation): how logistic regression finds parameters. MAP (Maximum A Posteriori): MLE with prior beliefs. Probability distributions: Gaussian (most ML assumes this), Bernoulli, Multinomial. The probabilistic perspective on ML.
Entropy: measure of uncertainty. Cross-entropy loss: the loss function for classification (why, not just what). KL divergence: measuring difference between distributions. MSE (Mean Squared Error): the loss for regression (derived from Gaussian assumption). Log loss, hinge loss (SVM). Understanding loss functions → understanding what models optimise → understanding model behaviour.
The skill that improves models more than algorithm choice — and the professional way to build ML workflows
Categorical encoding: one-hot, ordinal, target encoding (with regularisation to prevent leakage), binary. Numerical transforms: log transform (skewed data), polynomial features, binning. Date/time features: day of week, month, hour, is_weekend, days_since. Text features: TF-IDF, character n-grams, word count, sentiment score. Interaction features: feature1 × feature2. Feature engineering wins competitions and interviews.
Filter: correlation matrix, mutual information, chi-squared, variance threshold. Wrapper: Recursive Feature Elimination (RFE) — remove weakest feature, retrain, repeat. Embedded: L1 regularisation (Lasso — zeros out unimportant features), tree-based feature importance, permutation importance. Forward/backward selection. "I removed 60% of features and the model improved" — the power of feature selection.
Missing data strategies: drop (>50% missing), impute with mean/median (numerical), impute with mode (categorical), KNN imputer, iterative imputer (MICE). Missingness indicators: add binary column "was_missing." Outlier detection: IQR, Z-score, Isolation Forest. Outlier treatment: cap, winsorise, remove, separate model. "How do you handle missing data?" — the most asked feature engineering question.
Pipeline: chain preprocessing + model into one object (no data leakage). ColumnTransformer: different transforms for different column types (scale numeric, encode categorical). make_pipeline, make_column_transformer for quick creation. Custom transformers with BaseEstimator + TransformerMixin. Pipeline benefits: reproducible, testable, deployable. The professional way to build ML — not scattered fit/transform calls.
Train/validation/test split: why 3 sets, not 2. K-Fold cross-validation: more reliable than single split. StratifiedKFold: preserve class proportions (critical for imbalanced data). GroupKFold: prevent data leakage when samples are related. TimeSeriesSplit: respect temporal ordering. Nested cross-validation: tune hyperparameters without optimistic bias. "Why not just use train_test_split?" — because a single split is noisy.
Predicting continuous values — from linear regression to regularisation to advanced techniques
Cost function (MSE), normal equation (closed-form), gradient descent (iterative). Assumptions: linearity, independence, homoscedasticity, normality of residuals. Coefficient interpretation: "for each unit increase in X, Y increases by β." Multicollinearity: VIF (Variance Inflation Factor). Residual analysis: detecting violations. Not just sklearn.LinearRegression() — understanding what's happening inside.
Ridge (L2): penalise large coefficients → shrink towards zero but never exactly zero. Lasso (L1): penalise absolute values → coefficients CAN become exactly zero (automatic feature selection). Elastic Net: L1 + L2 combined. α (regularisation strength): too high → underfitting, too low → overfitting. When to use: Ridge (many useful features), Lasso (feature selection needed), Elastic Net (correlated features).
Polynomial regression: PolynomialFeatures → linear regression (linear in parameters, non-linear in features). Support Vector Regression (SVR): ε-tube, kernel trick for non-linear regression. Evaluation: R² (proportion of variance explained), adjusted R² (penalises extra features), MAE (interpretable), RMSE (penalises large errors), MAPE (percentage). Residual plots for model diagnostics. Bias-variance trade-off visualised through learning curves.
Predicting categories — logistic regression, trees, SVM, KNN, and Naive Bayes with internals
Sigmoid function: maps linear output to [0,1] probability. Log-odds (logit) interpretation. MLE for parameter estimation. Decision boundary: linear in feature space. Multiclass: One-vs-Rest, softmax. Coefficients as odds ratios. Regularised logistic regression (C parameter). "Explain logistic regression from scratch" — the question that tests ML depth, not sklearn familiarity.
Splitting criteria: Gini impurity (how often a random element would be misclassified), entropy/information gain. Recursive partitioning. Pruning: pre-pruning (max_depth, min_samples_split) vs post-pruning (cost-complexity). Advantages: interpretable, no scaling needed, handles non-linear. Disadvantages: overfit easily, unstable (small data change → different tree). CART algorithm. Feature importance from trees.
SVM: maximum margin, support vectors, kernel trick (linear → RBF → polynomial), soft margin (C parameter). KNN: lazy learner (no training), distance metrics (Euclidean, Manhattan, Minkowski), choosing K (odd, cross-validate). Naive Bayes: P(class|features) ∝ P(features|class)P(class), independence assumption (naive but works). Gaussian, Multinomial, Bernoulli variants. When each algorithm fits: SVM (binary, high-dim), KNN (small data, interpretable), NB (text, fast baseline).
Confusion matrix: TP, FP, TN, FN. Precision (of predicted positives, how many are correct), recall/sensitivity (of actual positives, how many are found). F1 = harmonic mean. Precision-recall trade-off: adjusting threshold. AUC-ROC: model's ability to discriminate. PR curve: better for imbalanced data. Calibration: are predicted probabilities meaningful? "Your precision is 0.95 but recall is 0.3 — what does this mean?" — the question that tests metric understanding.
Random Forest, XGBoost, LightGBM, CatBoost — the algorithms that win competitions and production
Bootstrap aggregating (bagging): train on random subsets, average predictions → reduces variance. Random Forest: bagging + random feature subsets at each split → decorrelated trees. Out-of-Bag (OOB) score: free validation estimate. Feature importance: mean decrease in impurity, permutation importance. Hyperparameters: n_estimators, max_depth, max_features, min_samples_leaf. RF is the "first algorithm to try" — robust, interpretable, hard to overfit.
Boosting: sequential ensemble where each tree corrects residuals of the previous. Gradient boosting: generalise to any differentiable loss function. Step-by-step: (1) fit tree to residuals, (2) multiply by learning rate, (3) add to ensemble, (4) repeat. Shrinkage (learning rate): smaller → more trees needed → better generalisation. Understanding gradient boosting internally is what enables tuning it effectively — "more trees + lower learning rate" isn't a random heuristic.
XGBoost: regularised gradient boosting (L1 + L2 on leaf weights), column subsampling, handles missing values natively, parallel tree construction. LightGBM: leaf-wise growth (faster, better accuracy), histogram-based binning, handles large datasets. CatBoost: native categorical feature support (no manual encoding), ordered boosting (reduced overfitting). When to use: XGBoost (default choice), LightGBM (large data), CatBoost (many categorical features). These three algorithms dominate tabular ML.
GridSearchCV: exhaustive (slow, thorough). RandomizedSearchCV: random sampling (faster, often sufficient). Optuna: Bayesian optimisation with Tree-structured Parzen Estimator (most efficient). Hyperparameter importance: learning_rate > n_estimators > max_depth > subsample > colsample_bytree. Early stopping: stop adding trees when validation score stops improving. Tuning with cross-validation (nested CV for unbiased estimates). The systematic approach to tuning, not random trial-and-error.
Stacking: train multiple diverse models (RF, XGBoost, LightGBM, logistic regression), use their predictions as features for a meta-learner. StackingClassifier/StackingRegressor in scikit-learn. Blending: simpler variant with holdout set. Diversity matters: ensemble of similar models doesn't help. Kaggle competition strategy: stack diverse models, tune the meta-learner. The technique that wins competitions and improves production models by 1–3%.
Clustering, dimensionality reduction, and finding structure without labels
K-Means: centroid-based, Elbow method, Silhouette score, k-means++ initialisation, limitations (assumes spherical, equal-size clusters). DBSCAN: density-based, no need to specify K, handles noise and arbitrary shapes. Hierarchical: agglomerative, dendrograms, linkage methods (Ward, complete, average). Gaussian Mixture Models: probabilistic clustering, soft assignments. Use cases: customer segmentation, anomaly detection, image compression.
PCA: project onto directions of maximum variance. Explained variance ratio, scree plot, choosing n_components (95% rule). PCA for denoising, compression, visualisation. t-SNE: non-linear, preserves local structure (great for visualisation, not for preprocessing). UMAP: faster than t-SNE, preserves global structure better. When to use: PCA (preprocessing, before KNN/SVM), t-SNE/UMAP (2D/3D visualisation of high-dimensional data).
Anomaly detection: Isolation Forest (anomalies are easier to isolate), One-Class SVM, autoencoders for anomaly detection (reconstruct normal → high reconstruction error = anomaly). Recommendation systems: collaborative filtering (user-user, item-item similarity), matrix factorisation (SVD), content-based (feature similarity). Use cases: fraud detection (banking), network intrusion (security), product recommendations (e-commerce).
Neural networks, CNNs, RNNs, and transfer learning — the algorithms powering modern AI
Perceptron → MLP. Neurons, weights, biases. Activation functions: ReLU (default hidden), sigmoid (binary output), softmax (multiclass). Forward pass → loss calculation → backpropagation → weight update. PyTorch: tensors, autograd, nn.Module, DataLoader. Training loop: zero_grad → forward → loss → backward → step. Building a classifier on tabular data from scratch in PyTorch.
Convolution: learnable filters detect features (edges → shapes → objects). Pooling: reduce spatial dimensions (max pooling). Architecture: Conv → ReLU → Pool → Flatten → Dense. Famous architectures: LeNet, VGG, ResNet (skip connections), EfficientNet. Transfer learning: use pretrained ImageNet model, replace final layer, fine-tune. torchvision for datasets and pretrained models. Practical: image classification, object detection overview.
RNN: process sequential data (time series, text) with hidden state. Vanishing gradient problem → LSTM (Long Short-Term Memory) with forget/input/output gates. GRU: simplified LSTM. Bidirectional RNNs. Sequence-to-sequence for translation. Attention mechanism: focus on relevant parts of input. Self-attention → Transformers. Why Transformers replaced RNNs: parallelisable, better long-range dependencies.
Batch normalisation: normalise layer inputs (faster training, regularisation). Dropout: randomly zero neurons during training (reduces overfitting). Data augmentation: create new training samples (flips, rotations, crops for images). Learning rate scheduling: reduce LR when loss plateaus. Early stopping. Optimisers: SGD with momentum, Adam, AdamW. Weight initialisation: Xavier, He. Mixed precision training for speed. The toolkit for training models that generalise.
Text processing, Hugging Face, LLM APIs, and the bridge to modern AI applications
Text preprocessing: tokenisation, lowercasing, stopwords, stemming, lemmatisation. Bag of Words, TF-IDF. Word embeddings: Word2Vec (CBOW, Skip-gram), GloVe. Sentiment analysis with scikit-learn (TF-IDF → logistic regression). Named Entity Recognition (NER) overview. Text classification pipeline: preprocess → vectorise → classify.
Transformer architecture: self-attention, multi-head attention, positional encoding. BERT: bidirectional, masked language model, fine-tuning for classification. GPT: autoregressive, next-token prediction. Hugging Face: pipeline() for zero-shot, text-classification, NER in 3 lines. Fine-tuning pretrained models: Trainer API, custom dataset. Hugging Face Hub: model sharing and versioning. The library that democratised NLP.
LLM APIs for feature engineering: generate text descriptions, extract entities, classify at scale. LLMs for data augmentation: generate synthetic training data. LLMs for labelling: zero-shot or few-shot classification on unlabelled data. Embeddings from LLMs (text-embedding-3-small) for similarity search. RAG overview: embed documents → vector store → retrieval → generation. The 2025–26 ML workflow integrates classical ML + LLMs.
SHAP, fairness metrics, and building models that stakeholders trust
SHAP (SHapley Additive exPlanations): game theory-based feature importance. SHAP values: contribution of each feature to each prediction. Summary plot: global feature importance. Waterfall plot: explain a single prediction. Dependence plot: feature effect with interactions. SHAP for XGBoost, Random Forest, neural networks. "Why did the model predict this customer will churn?" — SHAP answers this question. Essential for regulated industries (banking, healthcare).
Feature importance: built-in (tree-based), permutation (model-agnostic). LIME: local explanations (perturb inputs, observe changes). Partial Dependence Plots (PDP): feature effect averaged over dataset. Individual Conditional Expectation (ICE) plots. Accumulated Local Effects (ALE). Model-specific: coefficients (linear), attention weights (transformers). When to use: SHAP (best overall), LIME (quick local), PDP (simple feature effects).
Bias in ML: training data bias → model bias → discriminatory predictions. Fairness metrics: demographic parity, equalized odds, calibration by group. Detecting bias: disparate impact analysis, fairness auditing. Mitigation: resampling, reweighting, adversarial debiasing, post-processing. Privacy: differential privacy concepts, federated learning overview. EU AI Act awareness. "How would you ensure your model is fair?" — increasingly asked at responsible companies.
Experiment tracking, model serving, monitoring, and building a Kaggle+GitHub portfolio
Log every experiment: parameters, metrics, artifacts (model files, plots). Compare runs: which hyperparameters gave best AUC? Model registry: version models, stage transitions (staging → production). MLflow UI for visualisation. Why experiment tracking matters: reproducibility ("which model is deployed?"), collaboration ("what did you try?"), auditability. The MLOps foundation every ML team needs.
FastAPI: serve predictions via REST API (POST /predict with JSON body). Streamlit: interactive demos (sliders, file upload → prediction). Model serialisation: Pickle, Joblib, ONNX. Batch prediction vs real-time prediction. A/B testing in production: serve two models, compare metrics. The deployment path: train → serialise → serve → monitor.
Docker: package model + API + dependencies. CI/CD: GitHub Actions for automated training, testing, deployment. Cloud: AWS SageMaker (managed), Google Vertex AI, Azure ML. Student-friendly: Hugging Face Spaces (free model hosting), Railway, Render. "Here's my deployed model with a live API" — the strongest ML portfolio statement.
Kaggle competition approach: EDA → baseline → feature engineering → model selection → hyperparameter tuning → ensembling. Competition medals signal competence to employers. Kaggle notebooks: clean EDA with storytelling. GitHub repos: structured code, README, requirements. Portfolio: 4–5 projects showing progression (regression → classification → NLP → deployment → GenAI). Blog posts explaining projects. The portfolio IS the ML resume.
Feature engineering, handling imbalanced data, XGBoost + Optuna tuning, SHAP explainability, scikit-learn Pipeline, deployed via Streamlit.
Transfer learning with ResNet/EfficientNet on custom dataset, data augmentation, PyTorch training loop, Grad-CAM visualisation.
TF-IDF baseline → fine-tuned BERT with Hugging Face Trainer API. Compare traditional ML vs Transformer approaches.
Participate in an active Kaggle competition: EDA → feature engineering → model stacking → leaderboard submission. Document approach in notebook.
Trainers explain algorithms from the math up — gradient descent on a whiteboard, then implement in scikit-learn/PyTorch. Understanding WHY, not just HOW.
Weekly data challenges: build the best model on a real dataset. Leaderboard competition. Post-challenge review: what techniques won and why.
Trainers review: Pipeline structure, feature engineering choices, hyperparameter tuning approach, SHAP analysis, code quality. The feedback that builds ML engineering skills.
Students deploy their best model via FastAPI/Streamlit. Published on Kaggle with clean notebook. The portfolio that gets ML Engineer interviews.
ML Engineer, Data Scientist, AI Engineer — these roles at Google, Amazon, Flipkart, Goldman Sachs, and startups offer ₹12–30L+ packages. The demand for ML expertise far exceeds supply. Students who can build, evaluate, and deploy ML models are positioned for the highest-paying technical career path.
"Explain gradient boosting from scratch" "Derive the logistic regression loss function" "Why does L1 produce sparse solutions?" — ML interviews test DEPTH, not breadth. This module teaches algorithm internals, not just API calls. The student who can explain WHY XGBoost works outperforms the student who can only call XGBClassifier().fit().
ML hiring is portfolio-based: Kaggle profile, GitHub repos, deployed models. "Show me your best model" is how interviews start. Competition medals, clean notebooks, and deployed demos are the currency of ML hiring. This module builds 4–5 portfolio pieces on real data.
GenAI doesn't replace classical ML — it augments it. LLMs for feature engineering, data augmentation, and labelling. Classical ML for tabular prediction, time series, and structured data. The complete AI scientist knows BOTH. Module 8 (NLP + GenAI) bridges classical ML and modern LLM capabilities — the most valuable skill combination in 2025–26.
E-305, Industrial Area, Sahibzada Ajit Singh Nagar, Punjab 160055, IN
