Click any module to expand. This is NOT a "learn pandas" course — it's a complete data science training from statistics to ML to deployment. Every module includes hands-on work on real datasets. Module 2 (SQL) is a critical addition — SQL is tested in EVERY data science interview alongside Python.
NumPy, Pandas, Jupyter — the tools every data scientist uses daily
Jupyter Lab/Notebook: interactive coding + documentation + visualisation in one place. Markdown cells for documentation. Magic commands: %timeit, %%sql, %matplotlib inline. Google Colab for GPU access (free). Virtual environments with conda or venv. The data scientist's IDE — not VS Code, not PyCharm, Jupyter.
ndarray: creation (array, zeros, ones, arange, linspace, random). Indexing, slicing, boolean indexing. Vectorised operations: element-wise math without Python loops (50x faster). Broadcasting rules. reshape, flatten, transpose. Linear algebra: dot product, matrix multiplication (@), inverse, eigenvalues. Random number generation: np.random.default_rng(). NumPy is the foundation — Pandas, scikit-learn, and TensorFlow all use it internally.
Series and DataFrame creation. Reading data: read_csv, read_excel, read_json, read_sql. Indexing: loc (label), iloc (position), boolean. Column operations: adding, dropping, renaming. dtypes and astype for type conversion. .str accessor for string operations. .dt accessor for datetime. apply() with lambda for custom transformations. Method chaining for readable pipelines. Pandas IS the data science language — every analysis starts here.
groupby(): split-apply-combine. Aggregation: sum, mean, count, agg() with custom functions. merge(): SQL-style joins (inner, left, right, outer). concat() for stacking. pivot_table(): Excel-style summarisation with aggfunc. crosstab() for frequency tables. Multi-index DataFrames. "Group sales by region and month, calculate rolling average" — the daily Pandas workflow at analytics companies.
Missing data: isna(), dropna(), fillna() with mean/median/mode/forward-fill. Duplicates: duplicated(), drop_duplicates(). Outlier detection: IQR method, Z-score. Data type fixing: converting strings to dates, categories to codes. String cleaning: strip(), lower(), replace(). "80% of a data scientist's time is cleaning data" — this topic teaches the skills that consume most of the job.
The skill tested in EVERY data science interview alongside Python — you cannot skip SQL
SELECT with WHERE, ORDER BY, LIMIT. All JOIN types (INNER, LEFT, RIGHT, FULL). GROUP BY + HAVING for aggregation. Aggregate functions: COUNT, SUM, AVG, MIN, MAX. Subqueries: scalar, correlated. "Find the top 5 customers by total spend in each region" — the query pattern every data analyst writes daily. SQL is faster than Pandas for large datasets — and companies store data in databases, not CSV files.
ROW_NUMBER, RANK, DENSE_RANK for ranking. LEAD, LAG for accessing adjacent rows (month-over-month change). Running totals: SUM() OVER(ORDER BY date). Moving averages: AVG() OVER(ROWS BETWEEN 2 PRECEDING AND CURRENT ROW). NTILE for percentiles/quartiles. PARTITION BY for per-group calculations. Window functions ARE the language of analytics — Goldman Sachs, EXL, ZS Associates all test them heavily.
WITH clause for readable complex queries. Recursive CTEs for hierarchical data. CASE expressions for conditional logic. Date functions for time-based analysis. COALESCE for NULL handling. Query optimisation basics: EXPLAIN, indexes. Connecting SQL to Python: pd.read_sql() with SQLAlchemy. "Pull data from the database, clean in Python, build model in scikit-learn" — the real-world data science workflow.
The mathematical foundation — descriptive stats, distributions, hypothesis testing, and Bayesian thinking
Measures of central tendency: mean, median, mode — when to use each (mean for symmetric, median for skewed). Measures of spread: range, variance, standard deviation, IQR. Skewness and kurtosis. Percentiles and quartiles. Correlation: Pearson (linear), Spearman (monotonic) — correlation ≠ causation. Summary statistics with Pandas: describe(), corr(), value_counts().
Probability basics: conditional probability, Bayes' theorem, independence. Discrete distributions: Bernoulli, Binomial, Poisson. Continuous distributions: Normal (Gaussian — the most important), Uniform, Exponential. Central Limit Theorem: sample means are normally distributed regardless of population. Z-scores and standard normal table. Probability is the language of ML — every model outputs probabilities.
Population vs sample. Sampling distributions. Confidence intervals: "95% CI means if we repeated this 100 times, 95 intervals would contain the true parameter." Hypothesis testing: null hypothesis, p-value, significance level (α). Type I and Type II errors. t-test (one-sample, two-sample, paired). Chi-squared test for categorical data. ANOVA for comparing groups. A/B testing: "Did the new feature increase conversion?" — the business application of hypothesis testing.
A/B testing framework: control group, treatment group, metric, sample size calculation, duration. Statistical significance vs practical significance. Multiple comparison problem (Bonferroni correction). Bayesian A/B testing overview. "The new checkout flow increased conversion by 2.3% (p=0.03, 95% CI: [0.8%, 3.8%])" — how data scientists communicate results to business. Every product company runs A/B tests — data scientists design and analyse them.
Matplotlib, Seaborn, Plotly — and the structured EDA process that reveals insights
Matplotlib: figure, axes, plot, scatter, bar, hist, pie, subplots. Customisation: titles, labels, legends, colours, annotations. Seaborn: statistical plots built on Matplotlib — boxplot, violinplot, heatmap (correlation matrix), pairplot, countplot, displot, jointplot. When to use Matplotlib (full control) vs Seaborn (statistical, beautiful defaults). Saving publication-quality figures.
Plotly Express: interactive charts with one line of code. Hover tooltips, zoom, pan — for exploration. Plotly for dashboards (Dash framework). Choropleth maps for geographic data. Time series with rangeslider. When to use static (Matplotlib/Seaborn — reports, papers) vs interactive (Plotly — dashboards, presentations, exploration).
EDA is NOT random plotting — it's a structured investigation. The EDA framework: (1) Shape & size, (2) Data types & missing values, (3) Univariate distributions (histograms, value counts), (4) Bivariate relationships (scatter, box by category, correlation heatmap), (5) Outlier identification, (6) Feature-target relationship, (7) Key findings summary. EDA notebooks that tell a story — not a dump of 50 random charts.
The insight triangle: data → analysis → so what? (business implication). Structuring findings: context → insight → recommendation → expected impact. Choosing the right chart: bar (comparison), line (trend), scatter (relationship), pie (composition — use sparingly). Audience-appropriate communication: executive summary (1 slide) vs technical report (10 pages). "What did the data tell you?" — the question every interviewer asks. Having the technical skill without communication skill is half the job.
Transforming raw data into model-ready features — the skill that improves models more than algorithm choice
Label encoding (ordinal categories). One-hot encoding (nominal categories — pd.get_dummies, OneHotEncoder). Target encoding (mean of target per category — powerful but leaky if done wrong). Binary encoding for high-cardinality. When to use which: one-hot for tree models, target encoding for linear models. The encoding choice that makes or breaks model performance.
StandardScaler (z-score: mean=0, std=1) — for linear models, SVM, neural nets. MinMaxScaler (0–1 range). RobustScaler (handles outliers). When scaling matters: distance-based algorithms (KNN, SVM, K-Means), neural networks. When it doesn't: tree-based models (Random Forest, XGBoost). Always fit on training set, transform on test — never fit on test (data leakage).
Filter methods: correlation analysis, chi-squared, mutual information. Wrapper methods: Recursive Feature Elimination (RFE). Embedded methods: L1 regularisation (Lasso — coefficients go to zero), tree-based feature importance. SelectKBest for quick filtering. Why feature selection matters: fewer features = simpler model = less overfitting = faster training. The 80/20 rule: 20% of features carry 80% of predictive power.
Most real datasets are imbalanced (99% non-fraud, 1% fraud). Why accuracy is misleading: 99% accuracy by always predicting "non-fraud." Solutions: SMOTE (synthetic oversampling), random undersampling, class weights in models, stratified train-test split. Evaluation: precision, recall, F1, AUC-ROC (NOT accuracy). "How do you handle imbalanced data?" — asked in every ML interview.
Pipeline: chain preprocessing steps + model into one object. ColumnTransformer: different transforms for different column types (scale numeric, encode categorical). Pipeline benefits: no data leakage (fit_transform on train, transform on test automatically), cleaner code, easier deployment. make_pipeline for quick pipeline creation. The professional way to build ML workflows — not separate .fit() .transform() calls scattered through the notebook.
Regression, classification, and the algorithms that power predictive analytics
Linear regression: assumptions (linearity, no multicollinearity, homoscedasticity), coefficients interpretation, R² score. Regularisation: Ridge (L2), Lasso (L1), Elastic Net. Logistic regression: sigmoid function, log-odds, probability output, decision boundary. Multiclass: One-vs-Rest. These are the BASELINE models — always start here before trying complex algorithms. "Explain the difference between linear and logistic regression" — interview staple.
K-Nearest Neighbours: distance-based, no training phase, sensitive to scale, choosing K (odd to avoid ties). SVM: maximum margin classifier, kernel trick (linear, RBF, polynomial), support vectors. Naive Bayes: probability-based, assumes feature independence, works surprisingly well for text. When to use each: KNN (small datasets, interpretable), SVM (binary classification, high dimensions), Naive Bayes (text classification, spam filtering).
Decision tree: recursive splitting (Gini impurity, entropy), pruning, max_depth. Random Forest: ensemble of decorrelated trees (bagging + feature randomisation). Why RF beats individual trees: lower variance, more stable. Feature importance from RF. Hyperparameters: n_estimators, max_depth, min_samples_split. Random Forest is the "first algorithm to try" for most tabular data problems.
Gradient boosting: sequential ensemble where each tree corrects the previous. XGBoost: the Kaggle competition king — regularisation, handling missing values, GPU support. LightGBM: faster training (leaf-wise growth), handles large datasets. CatBoost: native categorical feature support. Hyperparameters: learning_rate, n_estimators, max_depth, min_child_weight, subsample. XGBoost/LightGBM wins 80%+ of tabular data Kaggle competitions — the most important algorithms to master.
Classification: accuracy, precision, recall, F1-score, AUC-ROC curve, confusion matrix, precision-recall curve. Regression: MAE, MSE, RMSE, R², adjusted R². Cross-validation: KFold, StratifiedKFold — never evaluate on training data. Train/validation/test split strategy. Bias-variance trade-off: underfitting vs overfitting. Learning curves for diagnosis. "Your model has 95% training accuracy and 70% test accuracy — what's wrong?" — overfitting, the answer every interviewer expects.
GridSearchCV: exhaustive search (slow but thorough). RandomizedSearchCV: random sampling (faster, often sufficient). Optuna: Bayesian optimisation (most efficient — finds best params with fewer trials). Halving search: progressive elimination. Cross-validation inside tuning (nested CV). Early stopping for boosting models. "How do you tune hyperparameters?" — the professional answer is Optuna or RandomizedSearchCV, not manual trial-and-error.
Clustering, PCA, and finding structure in unlabelled data
K-Means: centroid-based, choose K with Elbow method and Silhouette score. Initialization: k-means++. Hierarchical: agglomerative (bottom-up), dendrogram for choosing number of clusters. DBSCAN: density-based, handles noise and non-spherical clusters. Use cases: customer segmentation, anomaly detection, market basket analysis. "Segment these customers into groups" — the clustering interview question.
PCA: find directions of maximum variance, project data onto fewer dimensions. Explained variance ratio: how much information each component retains. Choosing number of components: 95% variance retention rule. t-SNE and UMAP for 2D visualisation of high-dimensional data. When to use: before KNN/SVM (reduce curse of dimensionality), visualisation, noise reduction. PCA is the most asked unsupervised learning interview topic.
Isolation Forest: tree-based anomaly detection (anomalies are easier to isolate). One-Class SVM. Statistical: Z-score, IQR. Use cases: fraud detection, network intrusion, manufacturing defects. Evaluation: precision/recall on labelled anomalies, visual inspection. "How would you detect fraudulent transactions?" — the anomaly detection interview scenario at banking/fintech companies.
Neural networks, CNNs, NLP fundamentals, and the bridge to GenAI
Perceptron → multi-layer perceptron (MLP). Neurons, weights, biases, activation functions (ReLU, sigmoid, softmax). Forward propagation → loss calculation → backpropagation → gradient descent. Epochs, batch size, learning rate. Keras/TensorFlow: Sequential model, Dense layers, model.compile(), model.fit(). The "Hello World" of deep learning: MNIST digit classification. Overview depth — not a deep learning specialisation.
Convolutional Neural Networks: convolution, pooling, feature maps. Why CNNs work for images: spatial hierarchy of features (edges → shapes → objects). Transfer learning: use pre-trained models (ResNet, VGG, EfficientNet) — fine-tune on your data instead of training from scratch. Practical: image classification with transfer learning in 20 lines of code. The overview that connects to computer vision careers.
Text preprocessing: tokenisation, lowercasing, stopword removal, stemming, lemmatisation. Bag of Words and TF-IDF for text representation. Sentiment analysis with scikit-learn. Word embeddings: Word2Vec, GloVe (conceptual). Hugging Face Transformers: pre-trained models for text classification, NER, summarisation in 5 lines of code. The bridge from traditional NLP to modern LLM-based approaches.
Time series components: trend, seasonality, residual. Stationarity: ADF test. Moving averages, exponential smoothing. ARIMA model (conceptual). Prophet by Meta for quick forecasting. Feature engineering for time series: lag features, rolling statistics, date features. Use cases: sales forecasting, demand prediction, stock price analysis. The overview that enables time series project work.
Using LLMs in data science workflows — the 2025–26 addition every employer expects
Calling LLM APIs from Python: openai.chat.completions.create(). Prompt engineering for data tasks: summarise this dataset, generate SQL from natural language, explain this model's output. Temperature, max_tokens, system prompts. Structured output: force JSON responses for programmatic use. Cost awareness: token counting, model selection (GPT-4 vs GPT-4o-mini for different tasks). The API that turns a data scientist into a 10x data scientist.
RAG: feed your own data to an LLM without fine-tuning. Architecture: embed documents → store in vector database (ChromaDB, Pinecone) → user query → retrieve relevant chunks → LLM generates answer from context. LangChain framework. Use cases: chat with your CSV data, query internal documentation, build domain-specific Q&A. The most in-demand GenAI pattern in 2025–26 enterprise AI.
LLM agents that write and execute code: "Analyse this dataset and find the top 3 insights." PandasAI: chat with your DataFrame. Code Interpreter pattern: LLM writes Python → executes → returns results. Limitations: hallucination, code errors, security (sandboxing). When to use AI agents (exploration, prototyping) vs manual analysis (production, critical decisions). The tool that amplifies data science productivity.
Serving models, tracking experiments, and building a portfolio that gets hired
Pickle and Joblib for saving scikit-learn models. ONNX for framework-agnostic model export. FastAPI: serve model predictions via REST API. Streamlit: build interactive ML demos in minutes (slider for inputs → real-time predictions). Gradio as alternative. The deployment path: train → save → serve via API → frontend calls API.
MLflow: log parameters, metrics, and artifacts for every experiment run. Compare runs: which hyperparameters gave best AUC? Model registry: version models, promote to production. Why experiment tracking matters: "Which model version is deployed?" "What hyperparameters were used?" — without tracking, you can't answer. MLflow is the experiment tracking standard — used at Databricks, Netflix, and most ML teams.
Docker: package model + dependencies + API into one container. Dockerfile for ML: Python base, install requirements, copy model + code, expose port. Cloud deployment: AWS SageMaker (managed ML), Google Vertex AI, Azure ML. Student-friendly: Railway, Render, Hugging Face Spaces (free). "Here's my deployed model with a live API" — the strongest data science portfolio statement.
Kaggle profile: competitions (even just participating signals interest), notebooks (well-documented EDA and models), datasets. GitHub repos: clean code, README with results, requirements.txt. Portfolio structure: 3–4 projects showing progression (EDA → ML → deployment → GenAI). Blog posts: explain your projects on Medium or personal site. The portfolio IS the resume for data scientists — "Show me your Kaggle profile" is how hiring works.
EDA on telecom dataset, feature engineering, XGBoost model with hyperparameter tuning (Optuna), SHAP for explainability, deployed via Streamlit with live predictions. Published on Kaggle.
Regression on real estate data, handling missing values and outliers, feature selection (RFE), pipeline with ColumnTransformer, Ridge/Lasso/RF/XGBoost comparison, cross-validation. Kaggle competition submission.
NLP pipeline: text preprocessing → TF-IDF → logistic regression baseline → Hugging Face transformer → LLM API for zero-shot classification. Compare approaches. Deploy best model via FastAPI.
Build a "chat with your data" application: upload CSV → embed with OpenAI → store in ChromaDB → query with natural language → LLM generates insights. LangChain + Streamlit frontend.
Trainers work on real datasets (Kaggle, UCI, company-donated) — not toy examples. Every concept demonstrated with data that has missing values, outliers, and messy reality.
Students compete on internal data challenges: clean data, build model, submit predictions, leaderboard ranking. Mimics real Kaggle competitions and company data science assessments.
Trainers review student EDA notebooks: is the story clear? Are visualisations informative? Are findings actionable? The feedback that transforms "I plotted some charts" into "I found an insight."
Students deploy their best model via Streamlit/FastAPI with a live URL. Published on Kaggle with a clean notebook. The portfolio that gets data science interviews.
Data Scientist, ML Engineer, Data Analyst — these roles at Goldman Sachs, EXL, ZS Associates, Mu Sigma, and product companies like Flipkart and Swiggy offer ₹8–20L starting packages. The demand-supply gap means qualified candidates get multiple offers. Python + SQL + ML + communication = the complete data science hiring checklist.
Banking (risk, fraud), healthcare (diagnostics), e-commerce (recommendations), telecom (churn), manufacturing (predictive maintenance), consulting (analytics). Data science isn't limited to tech companies — every industry is hiring. Students with data science skills have the widest career optionality of any technical specialisation.
LLM APIs, RAG systems, AI agents — the GenAI wave is creating new roles that combine traditional data science with LLM integration. "Data Scientist + GenAI" is the most in-demand profile in 2025–26. Module 9 (GenAI Integration) positions students at this intersection — the fastest-growing segment of tech hiring."
Data science hiring is portfolio-based, not certificate-based. Kaggle notebooks, GitHub repos, deployed models, and published analyses. "Show me your best project" is how interviews start. This module builds 3–4 portfolio pieces on real datasets — each demonstrating progressively advanced skills from EDA to ML to deployment to GenAI.
E-305, Industrial Area, Sahibzada Ajit Singh Nagar, Punjab 160055, IN
