Machine Learning


WS 2017





V3 + Ü1 (6 ECTS credits)
Note: This page is for a course from a previous semester.
Find a list of current courses on the Teaching page.
Course Dates:




Lecture/Exercise Mon, 10:15-11:45 UMIC 025
Lecture/Exercise Thu, 14:15-15:45 AH I

Lecture Description

The goal of Machine Learning is to develop techniques that enable a machine to "learn" how to perform certain tasks from experience.

The important part here is the learning from experience. That is, we do not try to encode the knowledge ourselves, but the machine should learn it itself from training data. The tools for this are statistical learning and probabilistic inference techniques. Such techniques are used in many real-world applications. This lecture will teach the fundamental machine learning know-how that underlies such capabilities. In addition, we show current research developments and how they are applied to solve real-world tasks.

Example questions that could be addressed with the techniques from the lecture include

  • Is this email important or spam?
  • What is the likelihood that this credit card transaction is fraudulent?
  • Does this image contain a face?


The class is accompanied by exercises that will allow you to collect hands-on experience with the algorithms introduced in the lecture.

There will be both pen&paper exercises and practical programming exercises based on Matlab (roughly 1 exercise sheet every 2 weeks). Please submit your hand written solutions of the exercises the day before the exercise class in the submission box at room 129, UMIC. Please submit your source code solutions through the L2P system.

We ask you to work in teams of 2-3 students.


For most part, the lecture will follow the book by Bishop. Additional topics are covered in Duda & Hart's book Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006 R.O. Duda, P.E. Hart, D.G. Stork, Pattern Classification, 2nd Edition, Wiley-Interscience, 2000 Wherever research papers are necessary for a deeper understanding, we will make them available on this web page.

Additional Resources

Matlab Resources

Course Schedule
Date Title Content Material
Introduction Introduction, Probability Theory, Bayes Decision Theory, Minimizing Expected Loss
Prob. Density Estimation I Parametric Methods, Gaussian Distribution, Maximum Likelihood
Prob. Density Estimation II Bayesian Learning, Nonparametric Methods, Histograms, Kernel Density Estimation
Prob. Density Estimation III Mixture of Gaussians, k-Means Clustering, EM-Clustering, EM Algorithm
Linear Discriminant Functions I Linear Discriminant Functions, Least-squares Classification, Generalized Linear Models
Exercise 1 Matlab Tutorial, Probability Density Estimation, GMM, EM
Linear Discriminant Functions II Logistic Regression, Iteratively Reweighted Least Squares, Softmax Regression, Error Function Analysis
Linear SVMs Linear SVMs, Soft-margin classifiers, nonlinear basis functions
Non-Linear SVMs Soft-margin classifiers, nonlinear basis functions, Kernel trick, Mercer's condition, Nonlinear SVMs
Model Combination Model Combination, AdaBoost, Exponential error, Sequential Additive Minimization
Exercise 2 Linear Discriminants, SVMs
Adaboost AdaBoost, Exponential error, Sequential Additive Minimization
Random Forests Randomized Decision Trees, Random Forests, Random Ferns
Exercise 3 AdaBoost, Random Forests
Neural Networks I Single-Layer Perceptron, Multi-Layer Perceptron, Mapping to Linear Discriminants, Error Functions, Regularization, Multi-layer Networks, Chain rule, Gradient Descent
Neural Networks II Backpropagation, Computational Graphs, Stochastic Gradient Descent, Minibatch Learning, Optimizers (Momentum, RMS-Prop, AdaGrad, Adam)
Tricks of the Trade Initialization (Glorot, He), Nonlinearities, Drop-out, Batch Normalization, Learning Rate Schedules
Convolutional Neural Networks I CNNs, Convolutional Layers, Pooling Layers, LeNet
Exercise 4 Softmax, Backpropagation
Convolutional Neural Networks II ImageNet Challenge, Notable Architectures, AlexNet, VGGNet, Inception, Visualizing CNNs
Tutorial Hands-on tutorial on TensorFlow
CNN Architectures & Applications I Residual Networks, Fully Convolutional Networks, Applications of CNNs
Word Embeddings Dealing with Discrete Data, Word Embeddings, word2vec, GloVe, Hierarchical Softmax, Motivation for Recurrent Networks
Recurrent Neural Networks I Plain RNNs, Backpropagation through Time, Practical Issues, Initialization
Exercise 5 Convolutional Neural Networks
Recurrent Neural Networks II LSTM, GRU, Applications of RNNs
Wrapping Up Current Research Directions in Deep Learning
Repetition Repetition
Exercise 6 RNNs
Disclaimer Home Visual Computing institute RWTH Aachen University