Machine Learning
Semester: |
WS 2019 |
Type: |
Lecture |
Lecturer: |
|
Credits: |
V3 + Ü1 (6 ECTS credits) |
Find a list of current courses on the Teaching page.
Lecture Description
The goal of Machine Learning is to develop techniques that enable a machine to "learn" how to perform certain tasks from experience.
The important part here is the learning from experience. That is, we do not try to encode the knowledge ourselves, but the machine should learn it itself from training data. The tools for this are statistical learning and probabilistic inference techniques. Such techniques are used in many real-world applications. This lecture will teach the fundamental machine learning know-how that underlies such capabilities. In addition, we show current research developments and how they are applied to solve real-world tasks.
Example questions that could be addressed with the techniques from the lecture include
- Is this email important or spam?
- What is the likelihood that this credit card transaction is fraudulent?
- Does this image contain a face?
Exercises
The class is accompanied by exercises that will allow you to collect hands-on experience with the algorithms introduced in the lecture.
There will be both pen&paper exercises and practical programming exercises (roughly 1 exercise sheet every 2 weeks). Please submit your solutions electronically through the RWTH Moodle system.
We ask you to work in teams of 2-3 students.
Literature
The first half of the lecture will follow the book by Bishop. For the second half, we will use the Deep Learning book by Goodfellow as a reference.
- Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006
- Ian Goodfellow, Yoshua Bengio, Aaron Courville, Deep learning, MIT Press, 2016
Wherever research papers are necessary for a deeper understanding, we will make them available in RWTH Moodle.
Additional Resources
- Kevin Murphy, Machine Learning -- A Probabilistic Perspective, MIT Press, 2012.
Python Resources
- A comprehensive python tutorial which is quite long
- Gives a very basic introduction to python and control loops (A sub topic of above link)
- This subsection gives an overview of python data structures such as list, dictionaries etc. (Again a sub topic of above link)
- A basic numpy tutorial
- PyCharm, a Python IDE
| Date | Title | Content | Material |
|---|---|---|---|
| Introduction | Introduction, Probability Theory, Bayes Decision Theory, Minimizing Expected Loss | ||
| Prob. Density Estimation I | Parametric Methods, Gaussian Distribution, Maximum Likelihood | ||
| Prob. Density Estimation II | Nonparametric Methods, Histograms, Kernel Density Estimation | ||
| Probability Density Estimation III | Mixture of Gaussians, k-Means, Clustering, EM-Clustering, EM-algorithm | ||
| Linear Discriminants I | Linear Discriminant Functions, Least squares Classification, Generalised Linear Models | ||
| Exercise 1 | Python Tutorial, Probability Density, GMM, EM | ||
| Linear Discriminants II | Logistic Regression, Iteratively Reweighted Least-Squares, Softmax Regression, Error Function Analysis | ||
| Linear Discriminants III | Softmax Regression | ||
| SVMs I | Support Vector Machines | ||
| SVMs II | Non-linear SVMs | ||
| Adaboost | Model Combination, AdaBoost, Exponential error, Sequential Additive Minimization | ||
| Neural Networks I | Single-Layer Perceptron, Multi-Layer Perceptron, Mapping to Linear Discriminants, Error Functions, Regularization, Multi-layer Networks, Chain rule, Gradient Descent | ||
| Exercise 2 | Linear Discriminants, SVM | ||
| Neural Networks II | Backpropagation, Computational Graphs, Stochastic Gradient Descent, Minibatch Learning, Optimizers (Momentum, RMS-Prop, AdaGrad, Adam) | ||
| Optimisation | Optimizers (Momentum, RMS-Prop, AdaGrad, Adam), Tricks of the Trade | ||
| Exercise 3 | AdaBoost | ||
| Convolutional Neural Networks I | CNNs, Convolutional Layers, Pooling Layers, LeNet | ||
| Exercise 4 | Softmax, Backpropogation | ||
| Tutorial | Tensorflow | ||
| Convolutional Neural Networks II | ImageNet Challenge, Notable Architectures, AlexNet, VGGNet, Inception, Visualizing CNNs | ||
| CNN Architectires and Applications | Residual Networks, Visualising CNNs, Applications of CNNs | ||
| Word Embeddings | Dealing with Discrete Data, Word Embeddings, word2vec, GloVe, Hierarchical Softmax, Motivation for Recurrent Networks | ||
| Recurrent Neural Networks I | Plain RNNs, Backpropagation through Time, Practical Issues, Initialization | ||
| Recurrent Neural Networks II | LSTM, GRU, Applications of RNNs | ||
| Exercise 5 | CNNs |
