Course in Foundations of Machine Learning
General Information
Lecturer  Prof. Dr. Martin Potthast 
Lab Advisors  Lukas Gienapp, Niklas Deckers 
Workload  2 SWS Lecture, 1 SWS Lab 
Language  Materials in English; lecture and lab in German; exam in German (but writing answers in English is possible). 
Requirements  See below: Requirements. 
Lecture  Tuesday, 13:15  14:45, HS 2 (Hörsaalgebäude), starting 10.10.2023. 
Lab  Each student is part of one of 4 groups (A, B, C or D), which determines the time slot for their lab session (Übung). A and B: On selected Wednesdays, 11:15  12:45, FelixKleinHörsaal (Paulinum, 5th floor). C and D: On selected Tuesdays, 11:15  12:45, FelixKleinHörsaal (Paulinum, 5th floor). The specific dates for the 6 sessions can be found below: Organization. 
Exam  Written exam at the end of the semester. 
Organization
 The lecture will take place in attendance. It will also be recorded and uploaded afterwards.
 In addition to the lecture, you will be provided with both theoretical and practical exercises. We will not collect or grade these exercises, but provide you with solutions and helpful tutorial videos (released at the same time). Nevertheless, you will still need to solve the exercises on your own/with a partner. You are responsible for your own learning success.
 We expect you to have studied the lecture, solved the exercises and reviewed the solutions before the corresponding lab session takes place. The lab sessions will be done in a flipped classroom format. This only works if you are well prepared. Please bring a laptop, tablet or smartphone to participate. The lab sessions will not be streamed or recorded.
The dates of your group's lab sessions can be found in the following table:Topic Group A Group B Group C Group D 1 25.10.2023 18.10.2023 24.10.2023 17.10.2023 2 08.11.2023 01.11.2023 14.11.2023 07.11.2023 3 29.11.2023 15.11.202328.11.2023 21.11.2023 4 13.12.2023 06.12.202312.12.2023 05.12.2023 5 10.01.2024 03.01.202409.01.2024 19.12.2023 6 24.01.2024 17.01.202423.01.2024 16.01.2024
We might be forced to close single groups (A, B, C, D) during the semester due to a lack of attendance. Remaining participants will then be asked to join another group.  Please also communicate with your fellow students to exchange about the course topics, the exercises and solution approaches.
 Lecture website  materials and announcements will be uploaded on this website.
 Email  important announcements will be sent out via email to students that are officially enrolled via AlmaWeb. Please notify Niklas Deckers immediately if you did not receive the welcome email at the start of the semester.
 Discord  you are invited to engage in discussions with your fellow students. We provide a Discord server that you are invited to join. The link will be provided in the welcome email.
Exam
 There will be a takehome mock exam that we are planning to release on 23.01.2024.
 It will be discussed in the lecture slot on 30.01.2024.
 The exam will take place on 06.02.2024, start at 12:45, in Audimax.
 The rewrite exam (Nachklausur) will take place on 26.03.2024, start at 11:00, in HS 1 (Hörsaalgebäude).
 The exam will be in German, but you are also allowed to write your answers in English.
 Please remember to bring your passport/Personalausweis and your student ID.
 You are allowed to bring a handwritten sheet (DIN A4, one side only) with useful information into the exam. It must be written by hand by yourself. The sheet will be collected together with the exam at the end of the exam.
 You are also allowed to bring a nonprogrammable calculator.
 If needed (international students), you may bring a dictionary.
 Mock exam, Mock exam solution (updated)
Requirements
Prerequisites for this course are the basic modules about algorithms and data structures, theoretical computer science and mathematics. We will use Python as a programming language.It might be helpful to refresh the following topics as their use will be required in the course. This list might be incomplete.
 Mathematical set notation
 Boolean formulas
 Matrix multiplication and transposition, matrixvector multiplication
 Partial derivatives, rules of derivation, Taylor formula
 Straight line equation: Formulating the equation, drawing and reading a plot; quadratic functions
 Laws of exponents, laws of logarithms
 Probabilities: Product rule, conditional probabilities, independence, total probability, Bayes Theorem
Lecturenotes
 Machine Learning » Introduction » Organization, Literature [video]
 Machine Learning » Introduction » Learning Tasks [video]
 Machine Learning » Introduction » Elements of Machine Learning [video]

Machine Learning »
Introduction »
Syntax & Model Overview
 Machine Learning » Machine Learning Basics » Concept Learning [video from this year (Niklas)] [video from last year]
 Machine Learning » Machine Learning Basics » From Regression to Classification [video]

Machine Learning »
Machine Learning Basics »
Evaluating Effectiveness [video part 1] [video part 2]
 Machine Learning » Linear Models » Logistic Regression [video part 1] [video part 2]
 Machine Learning » Linear Models » Overfitting and Regularization [video]

Machine Learning »
Linear Models »
Gradient Descent [video]
 Machine Learning » Bayesian Learning » Probability Basics [video]
 Machine Learning » Bayesian Learning » Bayes Classifier [video]

Machine Learning »
Bayesian Learning »
Frequentist versus Subjectivist
 Machine Learning » Decision Trees » Decision Trees Basics [video]
 Machine Learning » Decision Trees » Impurity Functions [video part 1] [video part 2]
 Machine Learning » Decision Trees » Decision Trees Algorithms [video]

Machine Learning »
Decision Trees »
Decision Trees Pruning [video]
 Machine Learning » Neural Networks » Perceptron Learning [video]
 Machine Learning » Neural Networks » Multilayer Perceptron [video part 1] [video part 2]
 Machine Learning » Neural Networks » Advanced MLPs [video]
Logbook
 10.10.2023: Until the end of Introduction > Elements of Machine Learning
 17.10.2023: Until the end of Machine Learning Basics > Concept Learning
 24.10.2023: Until slide ML:II155 in Machine Learning Basics > Evaluating Effectiveness
 07.11.2023: Until slide ML:III47 in Linear Models > Logistic Regression
 14.11.2023: Until the end of Linear Models > Gradient Descent
 21.11.2023: Until the end of Bayesian Learning > Bayes Classifier
 28.11.2023: Until slide ML:VI52 in Decision Trees > Impurity Functions
 05.12.2023: Until the end of Decision Trees > Decision Trees Pruning
 12.12.2023: Christmas exercise [notebook (Colab)] [recording]
 09.01.2023: Until slide ML:IV76 in Neural Networks > Multilayer Perceptron
 16.01.2023: Until the end of Neural Networks > Advanced MLPs
 23.01.2023: Q&A session [transcript]
 30.01.2023: Discussion of the mock exam and the Christmas exercise
Lab
The specific dates of the lab sessions can be found here: Organization.
Topic 1: Intro, Python and Maths Basics
 Relevant lectures: Until Introduction > Learning Tasks
 Homework (must be solved before your lab session):
 Exercise sheet 1 [solution  please review before the lab session] [explanatory videos: ex. 2  ex. 3c  ex. 4b  ex. 5a  ex. 5b  ex. 5c  ex. 5d]
 Practical exercise 1 [setup  video  requirements.txt] [python (ipynb)] [jupyter (ipynb)] [numpy (ipynb)  video]
 Lab exercises (will be solved by you during your lab session):
 Lab sheet 1 [solution  please only view after the lab session]
 PINGO  please only view after the lab session
Topic 2: Concept Learning & Evaluation
 Relevant lectures: Until Machine Learning Basics > Evaluating Effectiveness
 Homework:
 Exercise sheet 2 [solution] [explanatory videos: ex. 1a  ex. 1b  ex. 2a  ex. 2b]
 Practical exercise 2 [solution] [video]
 Lab exercises:
Topic 3: Linear Models
 Relevant lectures: Until Linear Models > Overfitting & Regularization
 Homework:
 Lab exercises:
Topic 4: Bayesian Classification
 Relevant lectures: Until Bayesian Learning > Bayes Classifier
 Homework:
 Exercise sheet 4 [solution] [explanatory videos: ex. 1  ex. 2  ex. 3]
 Practical exercise 4 [solution] [video]
 Lab exercises:
Topic 5: TreeBased Classification
 Relevant lectures: Until Decision Trees > Pruning
 Homework:
 Exercise sheet 5 [solution] [explanatory videos: ex. 1  ex. 2a  ex. 2b]
 Practical exercise 5 [solution] [video]
 Lab exercises:
Topic 6: Neural Networks
 Relevant lectures: Until Neural Networks > Multilayer Perceptron
 Homework:
 Exercise sheet 6 [solution] [explanatory videos: ex. 1  ex. 2  ex. 3]
 Practical exercise 6 [solution]
 Lab exercises: