Unsupervised Learning

203.4700 (3700)

Semester A

General Course Information


Meeting Times: Monday  10-12

Instructor: Dr. Rita Osadchy

e-mail: rita [at]cs [dot]haifa.ac.il
Office: Jacobs 410
­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­
­­­­­____________________________________________________________________________________________________________

Course Description

This  course will survey algorithms for unsupervised learning and high dimensional data analysis. The course will cover probabilistic/generative models of high dimensional data, such as  Gaussian mixture models, factor analysis,  probabilistic latent semantic analysis,  independent component analysis as well as  spectral methods for dimensionality reduction, including multidimensional scaling, Isomap, locally linear embedding, graph Laplacian methods and kernel PCA. The course will provide an introduction to probabilistic graphical models.

 

Recommended Prerequisites

The course assumes some basic knowledge of probability theory and linear algebra;
for example, you should be familiar with

Tutorials of the above topics.

_____________________________________________________________________

Announcements

____________________________________________________________________

Lecture Notes

Date

Topic

Lecture notes

Reading material

31.10

Introduction

PDF

Probability Tutorial PDF

MLE PDF

Bayesian Learning & Naive Bayes PDF

Parzen Windows PDF

Hierarchical clustering PDF

 

7.11

No class

 

14.11

Linear Dimensionality Reduction: PCA, MDS

PDF

Min Error Derivation of PCA PDF

MDS Tutorial: http://www.mathpsyc.uni-bonn.de/doc/delbeke/delbeke.htm

 

 

21.11

Manifold Mapping: Isomap, LLE

PDF

Isomap: http://isomap.stanford.edu/

LLE: http://cs.nyu.edu/~roweis/lle/papers/lleintroa4.pdf

Laplacian Eigenmaps: http://www.cse.ohio-state.edu/~mbelkin/papers/LEM_NC_03.pdf

28.11

Manifold Mapping -- Laplacian Eigenmaps.

KPCA

PDF

 

PDF

Laplacian Eigenmaps: http://www.cse.ohio-state.edu/~mbelkin/papers/LEM_NC_03.pdf

KPCA: http://cseweb.ucsd.edu/classes/fa01/cse291/kernelPCA_article.pdf

Comparison:  here

5.12

Kmeans,

Mixture of Gaussians, EM

PDF

PDF

 

Lecture notes: kmeans, MoG

12.12

EM derivation, EM for MoG

PDF

 

Lecture notes: EM

19.12

Cont. Laten Variables: Factor Analysis

PDF

 

Lecture notes: FA, Bishop PRML Ch. 12.2

26.12

PPCA,

LSI,

PDF

PDF

 

2.12

ICA

Intro to Graphical Models

Part1

PDF

PDF

Lecture notes: ICA

http://research.microsoft.com/en-us/um/people/cmbishop/prml/Bishop-PRML-sample.pdf

9.12

Intro to Graphical Models

Part I

PDF

http://research.microsoft.com/en-us/um/people/cmbishop/prml/Bishop-PRML-sample.pdf

16.12

Intro to Graphical Models part 2

PDF

http://research.microsoft.com/en-us/um/people/cmbishop/prml/Bishop-PRML-sample.pdf

23.12

HMM

PDF

Lecture notes: HMM

_________________________________________________________________

Textbook:

______________________________________________________________________

Probability tutorials:

 

http://www.autonlab.org/tutorials/prob18.pdf - first half

 

http://www-stat.stanford.edu/~susan/courses/s116/

 

Linear Algebra tutorial:

Eigen value decomposition

______________________________________________________________________