Online NPTEL Courses on Balaraman Ravindran

Introduction to Data Analytics

Mon, 01 Jan 0001 00:00:00 +0000

Data Analytics is the science of analyzing data to convert information to useful knowledge. This knowledge could help us understand our world better, and in many contexts enable us to make better decisions. While this is the broad and grand objective, the last 20 years has seen steeply decreasing costs to gather, store, and process data, creating an even stronger motivation for the use of empirical approaches to problem solving. This course seeks to present you with a wide range of data analytic techniques and is structured around the broad contours of the different types of data analytics, namely, descriptive, inferential, predictive, and prescriptive analytics.

This course is offered jointly by Prof. Nandan Sudarsanam. The tables below enlists the courses materials for Week 1 to Week 9. Each topic has both YouTube link and VideoKen link.

Week 1 - Course Overview & Descriptive Statistics

Course overview - 1
Course overview - 2
Descriptive statistics - graphical approaches
Descriptive statistics - measures of central tendency
Descriptive statistics - measures of dispersion
Assignment 1
Solution 1

Week 2 - Probability Distributions & Inferential Statistics

Random variables & probability distributions
Probability distributions - 2
Probability distributions - 3
Inferential statistics - motivation
Inferential statistics - single sample tests
Assignment 2
Solution 2

Week 3 - Inferential Statistics

Two sample tests
Type I & type II errors
Confidence intervals
ANOVA and test of independence
Short introduction to regression
Assignment 3
Solution 3

Week 4 - Machine Learning

Introduction to machine learning
Supervised learning
Unsupervised learning
Ordinary least squares regression
Simple & multiple regression in Excel & Matlab
Regularisation/coefficient shrinkage
Data modelling & algorithmic modelling approaches
Assignment 4
Assignment 4 data
Solution 4

Week 5 - Supervised Learning (Regression & Classification Techniques) I

Logistic regression
Training a logistic regression classifier
Classification & regression trees - 1
Classification & regression trees - 2
Bias variance dichotomy
Model assessment and selection
Support vector machines - 1
Support vector machines - 2
SVMs for non-linearly separable data
SVMs & kernel transformations
Assignment 5
Assignment 5 data
Solution 5

Week 6 - Supervised Learning (Regression & Classification Techniques) II

Ensemble methods & random forests
Artificial neural networks - 1
Artificial neural networks - 2
Deep learning
Assignment 6
Solution 6

Week 7 Association Rule Mining & Big Data

Association rule mining - 1
Association rule mining - 2
Big data - a small introduction - 1
Big data - a small introduction - 2
Assignment 7
Solution 7

Week 8 - Clustering Analysis & Predictive Analytics

Clustering analysis - 1
Clustering analysis - 2
Introduction to experimentation & active learning - 1
Introduction to experimentation & active learning - 2
Introduction to online learning - reinforcement learning - 1
Introduction to online learning - reinforcement learning - 2
Assignment 8
Solution 8

Week 9 - Course Summary

Course summary & insights into the final exam

Introduction to Machine Learning

Mon, 01 Jan 0001 00:00:00 +0000

With the increased availability of data from varied sources there has been increasing attention paid to the various data driven disciplines such as analytics and machine learning. In this course we intend to introduce some of the basic concepts of machine learning from a mathematically well motivated perspective. We will cover the different learning paradigms and some of the more popular algorithms and architectures used in each of these paradigms.

The tables below enlists the courses materials for Week 1 to Week 12. Each topic has both YouTube link and VideoKen link.

Week 1 - Introduction to Machine Learning & Probability Theory

A brief introduction to machine learning
Supervised learning
Unsupervised learning
Reinforcement learning
Probability tutorial - 1
Probability tutorial - 2
Assignment 1
Solution 1

Week 2 - Linear Algebra, Statistical Decision Theory, Linear Regression & Dimensionality Reduction

Linear algebra tutorial - 1
Linear algebra tutorial - 2
Statistical decision theory - regression
Statistical decision theory - classification
Bias-variance
Linear regression
Multivariate regression
Subset selection - 1
Subset selection - 2
Shrinkage methods
Principal components regression
Partial least squares
Assignment 2
Solution 2

Week 3 - Classification: Linear Models

Linear classification
Logistic regression
Linear discriminant analysis - 1
Linear discriminant analysis - 2
Linear discriminant analysis - 3
Weka tutorial
Assignment 3
Assignment 3 data
Solution 3

Week 4 - Optimisation & Classification: Separating Hyperplane Approaches

Optimisation tutorial
Perceptron learning
SVM - formulation
SVM - interpretation & analysis
SVMs for linearly non-separable data
SVM kernels
SVM hinge loss formulation
Assignment 4
Assignment 4 data
Solution 4

Week 5 - Artificial Neural Networks & Parameter Estimation

Early artificial neural network models
Backpropogation - 1
Backpropogation - 2
Initialisation, training & validation
Maximum likelihood estimate
Priors & MAP estimate
Bayesian parameter estimation
Assignment 5
Solution 5

Week 6 - Decision Trees

Introduction to decision trees
Regression trees
Stopping criteria & pruning
Loss functions for classification
Categorical attributes
Multiway splits
Missing values, imputation & surrogate splits
Instability, smoothness & repeated subtrees
Decision trees tutorial
Assignment 6
Assignment 6 data
Solution 6

Week 7 - Evaluation Measures & Hypothesis Testing

Evaluation measures
Bootstrapping & cross validation
2 class evaluation measures
The ROC curve
Minimum description length & exploratory analysis
Introduction to hypothesis testing
Hypothesis testing: basic concepts
Sampling distribution & the Z test
Student's T test
The two sample & paired sample T tests
Confidence intervals
Assignment 7
Solution 7

Week 8 - Ensemble Methods

Bagging, committee machines & stacking
Boosting
Gradient boosting
Random forest
Assignment 8
Solution 8

Week 9 - Graphical Models

Naive Bayes
Bayesian networks
Undirected graphical models - introduction
Undirected graphical models - potential functions
Hidden Markov models
Variable elimination
Belief propogation
Assignment 9
Solution 9

Week 10 - Clustering

Partitional clustering
Hierarchical clustering
Threshold graphs
The BIRCH algorithm
The CURE algorithm
Density based clustering
Assignment 10
Solution 10

Week 11 - Gaussian Mixture Models, Spectral Clustering & Learning Theory

Gaussian mixture models
Expectation maximisation - 1
Expectation maximisation - 2
Spectral clustering
Learning theory
Assignment 11
Solution 11

Week 12 - Frequent Itemset Mining & Reinforcement Learning

Frequent itemset mining
The apriori property
Introduction to reinforcement learning
RL framework & TD learning
Solution methods & applications
Miscellaneous: Multi-class classification
Assignment 12
Solution 12

Reinforcement Learning

Mon, 01 Jan 0001 00:00:00 +0000

Reinforcement learning is a paradigm that aims to model the trial-and-error learning process that is needed in many problem situations where explicit instructive signals are not available. It has roots in operations research, behavioral psychology and AI. The goal of the course is to introduce the basic mathematical foundations of reinforcement learning, as well as highlight some of the recent directions of research.

The tables below enlists the courses materials for Week 0 to Week 12. Each topic has both YouTube link and VideoKen link.

Week 0 - Preparatory Material

Probability tutorial - 1
Probability tutorial - 2
Linear algebra tutorial - 1
Linear algebra tutorial - 2
Assignment 0
Solution 0

Week 1 - Introduction to RL and Immediate RL

Introduction to RL
RL framework and applications
Introduction to immediate RL
Bandit optimalities
Value function based methods
Assignment 1
Solution 1

Week 2 - Bandit Algorithms

UCB 1
Concentration bounds
UCB 1 Theorem
PAC bounds
Median elimination
Thompson sampling
Assignment 2
Solution 2

Additional Reads

Auer, P.; Cesa-Bianchi, N.; Fischer, P. 2002. Finite-time Analysis of the Multiarmed Bandit Problem.

Auer, P.; Ortner, R. 2010. UCB Revisited: Improved Regret Bounds for the Stochastic Multi-Armed Bandit Problem.

Even-Dar, E.; Mannor, S.; Mansour, Y. 2006. Action Elimination and Stopping Conditions for the Multi-Armed Bandit and Reinforcement Learning Problems.

Tutorial on OFUL (Szepesvari, C.) Part 1 | Part 2 | Part 3

Week 3 - Policy Gradient Methods & Introduction to Full RL

Policy search
REINFORCE
Contextual bandits
Full RL introduction
Returns, value functions & MDPs
Assignment 3
Solution 3

Additional Reads

Notes on REINFORCE algorithm

Week 4 - MDP Formulation, Bellman Equations & Optimality Proofs

MDP modelling
Bellman equation
Bellman optimality equation
Cauchy sequence & Green's equation
Banach fixed point theorem
Convergence proof
Assignment 4
Solution 4

Week 5 - Dynamic Programming & Monte Carlo Methods

LPI convergence
Value iteration
Policy iteration
Dynamic programming
Monte Carlo
Control in Monte Carlo
Assignment 5
Solution 5

Week 6 - Monte Carlo & Temporal Difference Methods

Off Policy MC
UCT
TD(0)
TD(0) control
Q-learning
Afterstate
Assignment 6
Solution 6

Week 7 - Eligibility Traces

Eligibility traces
Backward view of eligibility traces
Eligibility trace control
Thompson sampling recap
Assignment 7
Solution 7

Week 8 - Function Approximation

Function approximation
Linear parameterization
State aggregation methods
Function approximation & eligibility traces
LSTD & LSTDQ
LSPI & Fitted Q
Assignment 8
Solution 8

Week 9 - DQN, Fitted Q & Policy Gradient Approaches

DQN & Fitted Q-iteration
Policy gradient approach
Actor critic & REINFORCE
REINFORCE (cont'd)
Policy gradient with function approximation
Assignment 9
Solution 9

Additional Reads

Notes on Policy Gradient Algorithms

Week 10 - Hierarchical Reinforcement Learning

Hierarchical reinforcement learning
Types of optimality
Semi-Markov decision processes
Options
Learning with options
Hierarchical abstract machines
Assignment 10
Solution 10

Additional Reads

Andrew G. Barto and Sridhar Mahadevan. 2003. Recent Advances in Hierarchical Reinforcement Learning. Discrete Event Dynamic Systems 13, 1-2 (January 2003), 41-77. DOI: https://doi.org/10.1023/A:1022140919877

Week 11 - Hierarchical RL: MAXQ

MAXQ
MAXQ value function decomposition
Option discovery
Assignment 11
Solution 11

Additional Reads

Dietterich, T. G. 2000. Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition.

Week 12 - POMDPs

POMDP introduction
Solving POMDP
Assignment 12
Solution 12

Additional Reads

POMDP Tutorial

Tutorial on Predictive State Representations (Singh, S. P.)